Friends, this is a cross-post of an article that was originally published on the Google Cloud blog on September 25, 2024 by Will Grannis, VP and CTO of Google Cloud. I wrote one response to a question from the LinkedIn community, and several of my OCTO colleagues contributed other answers as well. Enjoy, and please submit questions for future columns here.
What are the biggest real-world obstacles companies face when trying to implement and scale AI solutions?
Jeff Sternberg, Technical Director, Office of the CTO
Generative AI can increase implementation complexity since these models are inherently “creative” and therefore non-deterministic by default. Non-determinism means that each generation can produce slightly different results, such as different wording of a sentence or different pixels in a generated image. In highly regulated industries, such as financial services, this is particularly challenging as AI models must be explainable (both internally and to regulators), and organizations must be able to prove their outputs are correct. Nobody wants “hallucination” in a banking or payments transaction.
AI practitioners in these industries can mitigate the risks of non-determinism with strategies like grounding, which instruct models to base their outputs on authoritative reference information, such as real-time data from enterprise systems or policy documents. This context can be provided directly in the prompt itself or using techniques like retrieval augmented generation (RAG). Furthermore, you can adjust model parameters, such as temperature, to instruct the model to generate answers that are more factual.
It’s imperative to bring everyone in the organization together to learn the techniques being leveraged for model controls and governance and set up a positive feedback loop between teams.
Importantly, no matter which technical approaches are used to gain confidence in model behavior, a robust testing and validation system should be developed alongside the AI system itself. Evaluation is critical during development and should continue after production deployment, so that stakeholders and the AI product team can verify that the system is performing as expected over time. Logging is key — and don’t forget that AI tools can summarize logs and perform outlier detection to spot issues like model drift. It’s imperative to bring everyone in the organization together to learn the techniques being leveraged for model controls and governance and set up a positive feedback loop between teams.
Jeff Sternberg 