Function Calling and Java Integration with Spring AI Models

Spring AI is a powerful Spring Framework project that brings Java developers artificial intelligence (AI) capabilities. By integrating AI models into Java applications, Spring AI simplifies the process of creating intelligent applications while leveraging the robustness of the Spring ecosystem.

This article will guide you through the steps to integrate AI models into your Java application using Spring AI, with a special focus on the function-calling mechanism that allows your AI models to interact with external data sources and services dynamically.

Spring AI

Spring AI is designed to simplify the development of AI-powered applications in Java. Unlike some AI frameworks that are tightly coupled with Python, Spring AI is built to be language-agnostic, enabling Java developers to harness the power of AI without needing to switch to Python-based tools.

Spring AI draws inspiration from popular Python projects like LangChain and LlamaIndex, offering similar capabilities but tailored for the Java ecosystem. The framework is built with the assumption that the next wave of Generative AI applications will span multiple programming languages, and Spring AI aims to be a key player in this space.

How Does Function Calling Work?

Function calling in Spring AI allows your AI model to interact with external data sources or services during the processing of a request. This feature is particularly useful when your AI model needs information that it doesn't possess internally, such as real-time data from an external API.

For example, if your AI model needs to provide the current temperature in a specific location, it can invoke a function to call a weather API and retrieve this information. Spring AI simplifies the code required to implement this function-calling mechanism, making it easier to integrate dynamic data into your AI-driven applications.

Function Calling and Java Integration with Spring AI Models

Step 1: Add Spring AI Maven Dependencies

To get started, you need to include the necessary dependencies in your Maven project. These dependencies will allow you to work with Spring Boot, Spring Web, and TensorFlow for AI model integration.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.tensorflow</groupId>
    <artifactId>tensorflow</artifactId>
    <version>2.10.0</version>
</dependency>

Explanation:

• The spring-boot-starter dependency sets up a basic Spring Boot application.
• The spring-boot-starter-web dependency is used to create RESTful web services.
• The tensorflow dependency allows you to load and use TensorFlow models in your Java application.

Step 2: Load and Use the Model

Next, you'll need to load your AI model and prepare it for use. This step involves setting up a service class that manages the model's lifecycle and performs predictions.

import org.tensorflow.SavedModelBundle;
import org.tensorflow.Tensor;
import org.springframework.stereotype.Service;

@Service
public class ModelService {
    private SavedModelBundle model;

    // Constructor: Loads the AI model when the service is initialized
    public ModelService() {
        // Load the TensorFlow model from the specified path
        this.model = SavedModelBundle.load("path/to/model", "serve");
    }

    // Method: Makes predictions using the AI model
    public float[] predict(float[] input) {
        // Create a Tensor from the input data
        try (Tensor<Float> tensor = Tensor.create(input)) {
            // Run the model and get the output
            Tensor<?> result = model.session().runner().feed("input_tensor", tensor).fetch("output_tensor").run().get(0);
            // Convert the output Tensor to a float array and return it
            return result.copyTo(new float[1])[0];
        }
    }
}

Explanation:

• public ModelService(): Constructor that initializes the SavedModelBundle with the path to the TensorFlow model.
• public float[] predict(float[] input): Method that performs inference by creating a Tensor from the input data, running the model, and returning the result.

Step 3: Call the API from Spring

To make predictions, you'll often need to call a remote AI model API. This step involves creating a service that sends HTTP requests to the API and handles the responses.

import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

@Service
public class ApiService {
    private final RestTemplate restTemplate;

    // Constructor: Injects a RestTemplate for making HTTP requests
    public ApiService(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    // Method: Calls the remote AI model API to make a prediction
    public String predict(String input) {
        // The API endpoint where the model is hosted
        String apiUrl = "http://model-api-url/predict";
        // Send a POST request to the API and return the response as a String
        return restTemplate.postForObject(apiUrl, input, String.class);
    }
}

Explanation:

• ApiService class: This service handles communication with a remote API that hosts the AI model.
• RestTemplate: Used to send HTTP requests to the API.
• predict method: Sends the input data to the API and returns the prediction result.

Step 4: Create a REST Controller

Finally, you need to create a REST controller that exposes an endpoint for making predictions. This controller will handle incoming HTTP requests and interact with the ModelService.

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ModelController {
    private final ModelService modelService;

    // Constructor: Injects the ModelService
    public ModelController(ModelService modelService) {
        this.modelService = modelService;
    }

    // Endpoint: Handles POST requests to /predict
    @PostMapping("/predict")
    public float[] predict(@RequestBody float[] input) {
        // Use the model service to get a prediction and return it
        return modelService.predict(input);
    }
}

Explanation:

• ModelController class: This controller exposes an endpoint that clients can use to make predictions.
• @PostMapping("/predict"): Maps HTTP POST requests to the /predict URL to the predict method.
• predict method: Accepts input data in the request body, uses the ModelService to get a prediction, and returns the result.

Conclusion

In this article, we've explored how to integrate AI models into a Java Spring application using Spring AI. We covered the basics of function calling, loading AI models, making predictions, and exposing endpoints using REST controllers. By following these steps, you can easily add AI capabilities to your Java applications, making them smarter and more responsive to real-world data.