I am running a flask app accessed by endpoints on google app engine flex
There are certain modules that use shapely package of python. The exact
same app and inputs on my local machine for a certain calculation take 36
seconds say, while once deployed on GAE it takes above 9 to 10 minutes.
First I thought that locally often the IDE converts packages to cython
modules for speedups. I tried doing the same by converting the flex engine
to custom runtime, and running a Dockerfile to build the Cython packages.
My Dockerfile is as below
FROM gcr.io/google-appengine/python
LABEL python_version=python3.6
RUN virtualenv --no-download /env -p python3.6
# Set virtualenv environment variables. This is equivalent to running
# source /env/bin/activate
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
ADD requirements.txt /app/
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
ADD . /app/
RUN python cythonsetup.py build_ext --inplace
CMD exec gunicorn -t 600 -c gunicorn.conf.py -b :$PORT main:app
My cythonsetup.py to build and include the cython builds
from setuptools import setup,Extension
#from Cython.Build import cythonize
from Cython.Distutils import build_ext
ext_modules = [
Extension("addonModules.GridMakerCy", ["addonModules/GridMaker.py"], ),
Extension("addonModules.GeoCoordtoMetersCy",
["addonModules/GeoCoordtoMeters.py"],)]
for e in ext_modules:
e.cython_directives = {"language_level": "3"}
setup(name='GridMaker',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules,
)
And i import the built cythons correctly (I confirmed the code gave an
answer as I named the cython modules differently from python modules),
however the calc time didnt change -- locally it takes very little time
(even after including datastore and cloud storage read/writes--- which take
longer on a local machine with gcloud clients) but in the app engine it
takes very very long.
I logged into the GAE instance itself, and
after
docker exec -it gaeapp /bin/bash
i confirmed that cython is installed. Then why is the run so slow?
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/1d7a078d-c727-4f77-8a6f-6a474869ea87o%40googlegroups.com.
