initiation

Since the release of ChatGPT in November 2022, generative AI has been used by more and more people in a professional or private context. OpenAI also released an interface very early on to make their models available to developers. Back then, many companies carried out initial pilot developments based on the OpenAI interface.

The trend in generative AI has now developed towards countless providers making large, universal basic models (so-called foundation models) available commercially or on an open source basis. Major cloud providers such as AWS, Azure and Google Cloud now offer services to deliver these basic models in their own, secure cloud environment. The security, flexibility and scalability of the cloud, combined with open source basic models, opens up unprecedented opportunities for implementing solutions based on generative AI.

In this blog post, we would like to specifically look at the current options for deploying generative AI using the Amazon Bedrock AWS service.

Amazon Bedrock

Amazon Bedrock is a fully managed service that allows various current basic models to be delivered via one interface. Since this is a serverless service, the underlying computing infrastructure does not have to be managed by yourself.

Amazon Bedrock can be accessed either via the Internet or via a private connection directly from your own virtual private cloud (VPC). The latter is possible as soon as an interface endpoint is created via AWS Private Link.

The user-specific data (model inputs and outputs) is encapsulated, encrypted and not shared per user. In particular, this data is not used to improve the models. It is also possible to fine-tune models with your own data. In this case, the fine-tuned model is provided as a separate private instance (single tenant) to which only the creator has access.

In addition to using the basic models as standard, Amazon Bedrock currently has two options for adapting the large, universal models to your own data:

Retrieval Augmented Generation (RAG)

Retrieval Augmented Generation uses a generic basic model. The adaptation to customer data is context-based, without the need to adapt the model itself. The specific knowledge base that is to be added to the model is searched for input in a first step. Appropriate information is then given to the model as context in a second step so that a targeted output can be made.

Model adjustment

There are two types of model adjustment of basic models: fine-tuning and continued pre-training. During fine-tuning, model precision can be increased by using a task-specific labelled training data set to further adapt the basic model. Continued pre-training can be used to add more domain-specific knowledge to the basic model by using an unlabelled training data set to build knowledge for the basic model.

Amazon Bedrock also offers the option of building so-called agent systems using basic models. These are autonomous systems that orchestrate the interaction of basic models, data sources, software applications, and user input. They can be understood as programmed robots that can perform specific tasks independently and automatically. Agents thus go beyond the “next word prediction” principle of LLMs and now enable independent use of tools/applications. Agent systems are often attributed a high potential in the area of automation.

When using the models, you can choose between three modes: On-Demand, Batch, or Provisioned Throughput. In on-demand and batch mode, usage is calculated per input/output token; with provided throughput, the time units of deployment are calculated, regardless of usage.

Generative AI models on Amazon Bedrock

Amazon Bedrock supports various basic models from different providers. Currently, the offer is still quite different depending on the selected AWS region. For example, the current top model Anthropic Claude 3.5 Sonnet is currently only available in the North Virginia region (us-east-1). The following is an overview of the 8 best basic models that are currently available:

The basic models are sorted according to their results in CRFM Stanford's Massive Multi-task Language Understanding (MMLU) Benchmark. This is a common benchmark that aims to test the performance of LLMs in 57 subject areas with approximately 16000 multiple choice questions. The performance range of the basic models listed here is comparable to the well-known basic models GPT-3.5 Turbo (68.9%) and Gpt4o (84.2%) from OpenAI.

conclusion

Amazon Bedrock provides an easy-to-use solution for deploying generative AI. Thanks to the serverless architecture, the underlying computing infrastructure does not have to be managed by yourself. Comprehensive security mechanisms have been implemented to ensure the confidentiality and protection of your own data. State-of-the-art basic models are available in the areas of text and image, whose performance is comparable to GPT-3.5 or even Gpt4o from OpenAI.

In summary, Amazon Bedrock is an excellent opportunity to use generative AI and thus implement innovative solutions in the AWS cloud.