GitHub user logc opened a pull request:
https://github.com/apache/spark/pull/134
Add unit test to spark_ec2 script
Add a unittest for a refactored function to destroy cluster. Relies on
mock and moto dependencies to avoid sending out EC2 requests.
We noticed at work that the supplied script does not work always when
destroying clusters, specially for regions outside "us-east-1". This pull
request adds a test on the command. If this is accepted, we can continue
debugging why exactly are such clusters not terminated correctly.
Run the test with:
```console
$ python ec2/tests.py
Searching for existing cluster cluster_name...
Terminating master...
Terminating slaves...
Deleting security groups (this will take some time)...
Attempt 1
.
----------------------------------------------------------------------
Ran 1 test in 1.040s
```
Please notice that the test relies on
[moto](https://github.com/spulec/moto) and
[mock](https://pypi.python.org/pypi/mock) to run. Since I did not know how you
handle such dependencies, I have not added a `requirements.txt` or a `setup.py`
script, but can update the pull request in that sense.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/logc/spark tlpmaster
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/134.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #134
----
commit 2e7ca995dce6a8822fed4389be4d69223bd671d2
Author: Luis Osa <[email protected]>
Date: 2014-03-12T15:32:30Z
Add unittest for destroy cluster command
Add a unittest for refactored function to destroy cluster. Relies on
mock and moto dependencies to avoid sending out EC2 requests.
commit 534edcc96e33889275ad278dc3cfaf24faf44604
Author: Luis Osa <[email protected]>
Date: 2014-03-12T15:34:13Z
Add pyc files to gitignore
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
