Haseeb is CEO and co-founder of Rafaya leading platform provider for Cloud and Kubernetes Automation.
Artificial intelligence (AI) – specifically Generative AI (GenAI) – has quickly become a fundamental component in the design and development of new capabilities delivered by all software teams. GenAI gives developers superpowers to build modern applications that delight end users and uncover the business value business leaders desire. As groundbreaking as GenAI-powered solutions can be, technology leaders are trying to understand how to implement GenAI quickly and securely. Many teams face operational challenges: for example, quickly rolling out testbeds for developers to experiment with creating co-pilots can result in a huge cloud bill if controls aren’t put in place. Platform engineering teams can help you solve this.
Just as platform engineering teams play a critical role in accelerating the modernization of their applications, they can help companies adopt GenAI safely and easily by applying the same control- and efficiency-based techniques. With a platform approach to GenAI adoption, enterprises can develop new applications with GenAI capabilities, allowing developers and data scientists to develop quickly with the right guardrails in place.
Before the AI hype, companies began deploying cloud technologies to accelerate the development of modern applications, improve end-user experiences, and stay competitive. However, provisioning and managing cloud environments and Kubernetes is becoming difficult on a large scale, slowing the pace at which companies can innovate. To solve this dilemma, platform teams have been formed to effectively manage these environments, offering self-service consumption models that help developers deliver new capabilities faster.
Because GenAI infrastructure consumption (GPUs, vector databases, etc.) also occurs across public and private clouds, the same techniques that platform teams have applied to make app modernization initiatives successful through the self-service consumption of cloud resources can be applied to consumption of GenAI-related infrastructure. Platform teams can deliver workbenches for various GenAI use cases, such as augmented generation (RAG) retrieval and model refinement, to developers and data scientists, leading to faster, more advanced applications with governance and cost-related guardrails.
A word of caution for IT leaders: Speed is critical in GenAI adoption, and data scientists won’t wait for platform teams to develop workflows in-house. Challenge your platform teams to explore best-in-class solutions designed for cloud, Kubernetes, and environmental automation use cases.
How platform teams accelerate AI experiments
With access to the right tools, platform teams can enable developers and data scientists to move quickly with their GenAI experiments and production, while enforcing the necessary guardrails to ensure enterprise-level governance and control. To help their employers accelerate GenAI journeys, there are several areas that platform teams should invest in:
Composite templates: Made available by platform teams to developers and data scientists to ensure that “golden paths” are followed by all consumers of GenAI-based infrastructure and tooling, curated templates can help simplify environment configuration for deploying GenAI-based applications. In this way, platform teams can take the lead in controlled and efficient use of GenAI infrastructure, apps and tools. Developers and data scientists can also easily experiment with services like Amazon Bedrock, Microsoft Azure OpenAI, and ChatGPT to create natural language interfaces, chatbots, content generation systems, and more, without being burdened with infrastructure-related complexities.
Sample code: Platform teams can facilitate developers’ and data scientists’ experimentation with GenAI by providing well-documented and accessible code for a variety of GenAI use cases. Along with sample code, step-by-step guides for launching test environments, integrating the sample code with native apps, and running sample applications can greatly help developers and data scientists jump-start their GenAI journeys.
Self-service consumption: By providing developers and data scientists with self-service workflows to launch GenAI workbenches for different use cases (RAG, RHLF, etc.), they can independently pace their GenAI experiments. With the right combination of templates, sample code, and self-service workflows, platform teams can deliver on-demand environments with all the necessary dependencies and tools for different GenAI use cases across environments, accelerating the pace of experimentation and production. high-quality delivery of GenAI capabilities. The faster developers and data scientists can move, the faster the company will widen the gap between competitors.
Management: Platform teams, as part of the curated templates they can make available to developers and data scientists for self-service consumption of the GenAI infrastructure, can help define and enforce governance policies for data privacy, ethical considerations, and industry regulatory compliance, so as to stay safe and responsible GenAI experiments can take place. These compliance checks can be automated within CI/CD pipelines to ensure GenAI models adhere to their organization’s policies before deployment.
Access controls: It is important to implement fine-grained access controls to limit access to sensitive data and resources that may be exposed when building GenAI applications. Platform teams should be able to easily ensure that developers and data scientists have the appropriate permissions based on their roles and responsibilities.
Manage costs: Consuming GenAI-related infrastructure and tools can be incredibly expensive. GPUs and specialized base models are incredibly expensive. Platform teams must track and manage costs associated with GenAI experiments. Budget constraints should be a programmable attribute, along with workbench time-to-live policies, among other things. Cost management capabilities keep the overall investment in GenAI-related R&D in check while developers and data scientists can move quickly.
Overcoming the GenAI craze
Companies everywhere are experimenting with GenAI to learn and internalize how it can positively impact the business. However, the complexity of cloud and GenAI technologies has created an IT challenge. Platform teams within the broader IT organization can help overcome these GenAI-related challenges so that real innovation can occur, leading to a net positive impact on the end-user experience. As they have done before, platform teams should provide GenAI-ready workbenches in a self-service format to developers and data scientists so they can explore how to quickly deliver GenAI-based capabilities to their users. Companies should thoroughly evaluate and select vendors that enable platform teams to deliver self-service capabilities to developers and data scientists. Without platform teams in the mix of these new technologies, many GenAI-related experiments are doomed to failure and cost ridiculous amounts of money.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Am I eligible?