Chapter 5 Deploying to ECS

[KansaiUser]

Are you sure the application in the microservices/mlewp2-web-service is correct?
I did everything you write in the book (chapter 5 hosting on ECS) and it fails.

After investigating, and asking ChatGPT and other sources, they tell me that for a microservice to be host on AWS it has to have healthcheck endpoints. and your application does not seem to have them.

something like

from flask import Flask

app = Flask(__name__)

@app.route('/')
def home():
    return 'Hello, ECS!', 200

@app.route('/health')
def health():
    return 'Healthy', 200

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

[OmarAlmusa]

Yeah I had an issue with deploying this app as well, the healthchecks keeps saying service basic-ml-service port 5000 is unhealthy in target-group basic-ml-microservice-lb-tag due to (reason Health checks failed with these codes: [404]).

I've been trying to solve this issue for 5 hours and, and still couldn't solve it

[OmarAlmusa]

I found the solution. @KansaiUser thanks to you, as you mentioned we should create a new route for /health

in resources folder -> forecast.py:
add the following class at the end of the file:

class HealthCheck(Resource):
    def get(self):
        return {"status": "healthy"}, 200

then in app.py:
add HealthCheck to

from resources.forecast import ForecastHandler, Forecaster, HealthCheck

and then add api.add_resource(HealthCheck, '/health') after the line api.add_resource(ForecastHandler, '/forecast', resource_class_kwargs={'forecaster': forecaster})

app.py should look like this:

from flask import Flask
from flask_restful import Api, Resource
from resources.forecast import ForecastHandler, Forecaster, HealthCheck
import logging

app = Flask(__name__)
api = Api(app)

forecaster = Forecaster()
api.add_resource(ForecastHandler, '/forecast', resource_class_kwargs={'forecaster': forecaster})
api.add_resource(HealthCheck, '/health')

if __name__ == '__main__':
    logging.basicConfig(filename='app.log', level=logging.INFO, format='%(asctime)s | %(name)s | %(levelname)s | %(message)s')
    logging.info('Main app sequence begun')
    app.run(debug=True, host='0.0.0.0', port=5000) # change debug=False in production
    logging.info('App finished')

**IMPORTANT: don't forget to build the docker container after adding the lines, and also re-push to ECR after building the container image. If you are pushing with tag :latest then you should have no problem, but if with different tag eg. :v1...etc then keep in mind to change the tag in task definition as well.

after you finish pushing to ECR, go to ECS and then clusters, delete the old service, and start a new one, when you reach the load balancing section, in health check path, make sure you type '/health' and then wait for creation to finish. You should have healthy checks passed.

**IMPORTANT: After the service finishes creating and everything seems fine, if you go to Postman and request POST to the DNS provided by the loadbalancer, you might face issue where you don't receive or the request times out... this issue is related to security groups. To fix this:

Go to Clusters->Services->
and then click on "Configuration and networking" then scroll down to "Network configuration" you'll see the associated security group, click on it, it will redirect you to the security group that the service is using, then at the bottom you will see "Inbound rules" to the right there is button "Edit inbound rules" click on it then do the following:

Click "Add rule".

Set Type to HTTP.

Set Protocol to TCP.

Set Port range to 80 (or whatever port your service runs on).

Set Source to 0.0.0.0/0 (allows public access) or restrict it to your IP (My IP).

Click "Save rules".

and try sending post request to the DNS again, it should work.

[bit-scientist]

Hello, @OmarAlmusa. Sorry for the off-topic question: As someone who's actually read the book, how do you like it in terms of relevance, explanation and execution flow?

[OmarAlmusa]

Honestly I'd give it 8/10, it does introduce the tech stack that is necessary and used today in the industry, also emphasizes to look at ML Engineering from multiple perspectives, specifically the business perspective, which is crucial and overlooked when it comes to building ML workflows (people focus more on the cool AI frameworks and how the model should be perfect... when in reality understanding what the client wants and how the ML model would impact their business is far more important).

The major drawback of the book is that some details about the frameworks are outdated or lacked explaining the important details of code lines and relied heavily on their github repo to explain the concepts... which I had to go back and forth from book to github code to understand exactly what I am doing...

Ofcourse I faced some problems when applying their examples, like the one you see here in this issue, running the simple ML service in AWS, or even running Ray Framework on my local machine (couldn't train the models in parallel).

But overall it wasn't waste of time... I did benefit from the book and learned new things.

[KansaiTraining]

Even though nobody asked me (LOL) as the author of this issue I would have to say that this book is one of the best books I have read on the theme. I liked the first edition (I was in the middle of reading it when I got the second one) but the second edition improves it even more.

As Omar said, it has drawbacks. On my part, I also found things that required some effort to make them work, the code is sometimes not in the final state (I think the first edition had a whole chapter without code- I vaguely remember- but the code was in the second edition) but all in all the book provided a solid foundation on techniques and concepts as it promised. I recommend this book

[bit-scientist]

Thank you both @OmarAlmusa and @KansaiTraining. Appreciated your honest feedback

[bit-scientist]

@KansaiTraining (or @KansaiUser ?) , @OmarAlmusa I also have reached up to this point and faced the exact same issue. I somehow remembered that this issue had been mentioned here, but still tried to solve it myself. All in all, I came to the same way of solving the issue (yet to implement). I will try redoing the ch05 from scratch again.

@AndyMc629, please consider working on such issues if you're ever planning to publish the next edition in the future. This book has much to offer, yet it still misses important steps when beginner/elementary level readers are assumed.