Have you ever wondered how AI-based mobile applications work and how those apps can handle millions of users? The first part of the answer is “the cloud,” and the second part is scalability. Mobile applications you install on your mobile phone are usually just a user interface to a more complex system, living in “the cloud”. The cloud refers to servers accessed over the Internet and the software and databases that run on those servers. Cloud servers are located in data centers all over the world. Using cloud computing, users and companies do not have to manage physical servers or run software applications on their own machines. In our case, the cloud does all the hard work, running our heavy and complex machine learning models. In the case of mySmartBlood, the mobile app is an interface to enter blood tests and display the results of the analysis. Between these two steps, the app communicates with our cloud-hosted AI models, and this is where the magic happens. 

So now that we know what the cloud is, let’s see how it can handle all the workloads from all our users. Modern technologies used by cloud providers combined with sophisticated software frameworks allow us to do marvelous things that help our apps adapt to user needs. One such feature is autoscaling which helps us address the scalability requirement. AI models often require large amounts of memory (RAM) and computing time. Imagine that one ML model can only serve 50 users in a minute. If 200 users use the app simultaneously, 150 of them will spend lots of time waiting – which isn’t user-friendly.  But, by using microservice architecture, containerized deployments, and Kubernetes clusters, we can tell the cloud to set up additional servers and run our ML models on them when there are more active users. Of course, these instructions to the cloud have to be clearer, for example when the CPU demand is higher than 90% for 5 minutes, start preparing more servers and balance the work between them. This kind of thing used to be managed manually and took hours or even days, but it can now be automated and handled in minutes or seconds with modern technology and scalable cloud-based infrastructures.