![]() You can see that it installs some of the products that normally you use with Spark and Hadoop, like: What goes into EMR? Here is the configuration wizard for EMR. ![]() But it might cost less to have Airflow install all of that and then tear it down than the alternative: you leaving idle hardware running, especially if all of that is running at Amazon, plus perhaps paying a big data engineer to write and debug scripts to do all of this some other way. ![]() All the products it installs are open source. The point here is that you don’t need Airflow. So, it might be better to use Airflow than the alternative: typing spark-submit into the command line and hoping for the best.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |