Blogs

Artificial Intelligence Engineering – The ultimate survival guide (Part 1 of 3)

Artificial Intelligence (AI) and Machine Learning (ML) are undergoing a major revolution. The future of AI and ML is here, and now.

As various solutions are being launched in the market to ease the model development process the technology for AI Engineering is still getting off its infancy stage. Organizations are accruing huge technical debts to deploy models in production and struggling to make the process easier.

Eclipsys recently completed a project on deploying and automating machine learning models for an entertainment client with a focus on reducing Time to Value for such deployments. Artificial Intelligence and Machine Learning are embedded in their organization however like other companies they faced several challenges:

• Inflexible development and deployment processes plus unsupportive current enterprise data infrastructure stifling experimentation and collaboration and rapid deployments
• Longer time to market
• Lack of automation to detect drifts in model scores as new data is ingested
• No concept of baseline model sharing, versioning, data featurization or automating machine learning pipelines to reduce effort duplication
• Manual process for model explainability and model interpretation post model re-train
• Future scalability and increased data volume considerations pose challenges across data access, preparation and management
• ML engineers are unable to provide workspaces and instances to data scientists for ML development and model experiments with data mimicking the production
• ML Engineers also need able to manage and scale infrastructure for multiple training runs of models in production clusters

What does this mean? 

Most importantly, they have a backlog of model deployment that is hindering their ability to make strategic business decisions. Plus they had mounting issues due to a lack of adequate model training and even some incorrect predictions. 

Our customer was acutely aware of the operational inefficiencies and the accumulating technical debt as a result. Needing to do more with less allows a company to turn AI. An investment in AI Engineering and automation is the solution. Eclipsys designed a complete ML Automation solution architecture on Microsoft Azure Cloud Services. 

Coming soon!

Stay tuned for part 2 of our Artificial Intelligence Engineering blog series to learn the Microsoft Azure ML Engineering automation solution we designed. And part 3 where we showcase our Google solution.

More from this Author