Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAI: migrate to HttpClient #2529

Merged
merged 26 commits into from
Feb 13, 2025
Merged

OpenAI: migrate to HttpClient #2529

merged 26 commits into from
Feb 13, 2025

Conversation

dliubarskyi
Copy link
Member

@dliubarskyi dliubarskyi commented Feb 11, 2025

Issue

Closes #2468

Changes

  1. This PR is a continuation of efforts started in Customizable HTTP clients #2413
  2. When using langchain4j-open-ai, all OpenAi*Models are now using java.net.http.HttpClient instead of OkHttp/Retrofit by default. You also have an option to customize it or plug in any other HTTP client.
  3. When using langchain4j-open-ai-spring-boot-starter, all OpenAi*Models are now using Spring's RestClient instead of OkHttp/Retrofit by default. You also have an option to customize it or plug in any other HTTP client.
  4. apiKey is now optional for all OpenAi*Models
  5. OpenAi*Models can now throw dev.langchain4j.exception.HttpException instead of dev.ai4j.openai4j.OpenAiHttpException
  6. The default connect timeout is now 15 seconds instead of 60 for all OpenAi*Models. The default read timeout stays the same (60 seconds).
  7. To continue using the "demo" key, you now need to specify the baseUrl explicitly:
OpenAiChatModel model = OpenAiChatModel.builder()
        .baseUrl("http://langchain4j.dev/demo/openai/v1")
        .apiKey("demo")
        .build()
  1. If you are using Proxy, now you need to configure it directly on the HTTP client of your choice. For example, when using JDK's HttpClient:
HttpClient.Builder httpClientBuilder = HttpClient.newBuilder()
        .proxy(ProxySelector.of(new InetSocketAddress("XXX.XXX.XXX.XXX", 1234)));

OpenAiChatModel model = OpenAiChatModel.builder()
        .httpClientBuilder(JdkHttpClient.builder().httpClientBuilder(httpClientBuilder))
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o-mini")
        .build();

This also means that all langchain4j.open-ai.*-model.proxy.* Spring Boot properties are not working any more.
9. OpenAiEmbeddingModel: removed default modelName (TEXT_EMBEDDING_ADA_002), please set it explicitly now
10. OpenAiImageModel: removed withPersisting and persistTo properties. Persisting images will not be supported any more.
11. OpenAiLanguageModel and OpenAiStreamingLanguageModel: removed default modelName (GPT_3_5_TURBO_INSTRUCT) and temperature (0.7), please set it explicitly now
12. OpenAiModerationModel: removed default modelName (TEXT_MODERATION_LATEST), please set it explicitly now
13. All the OpenAi*Model constructors are now accepting a builder object instead of all the properties
14. The langchain4j-local-ai module is now also using java.net.http.HttpClient instead of OkHttp/Retrofit as it depends on the langchain4j-open-ai module

General checklist

Sorry, something went wrong.

WIP

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The migration from OpenAI4j to a custom HttpClient might introduce subtle differences in request/response handling, potentially affecting existing integrations that rely on specific OpenAI4j behaviors.
  • Removal of proxy configuration from the OpenAiClient builder. Users now need to configure the proxy in the HttpClientBuilder.

Potential Design Issues

  • The introduction of a custom HttpClient adds complexity to the project and might require more maintenance effort compared to relying on a well-established library like OpenAI4j.
  • The design might be less flexible if the custom HttpClient lacks features or customization options available in other HTTP client libraries.

Potential Bugs

  • The SSE implementation in the custom HttpClient might not be as robust as the one in OpenAI4j, potentially leading to issues with long-running streams or error handling.
  • The custom HttpClient might not handle all edge cases related to HTTP headers, request/response encoding, or error handling, potentially leading to unexpected behavior.
  • The removal of callTimeout, connectTimeout, readTimeout, and writeTimeout from the OpenAiClient builder and relying on the HttpClientBuilder's timeouts might lead to unexpected timeout behavior if the HttpClientBuilder's defaults are different.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and request are provided.
    • When: Provide a correct API key and a valid request in the OpenAiClient builder.
    • Then: The HttpClient should successfully connect to the OpenAI API and return the expected response.
    • Given: SSE streaming is enabled.
    • When: Set stream to true in the ChatCompletionRequest and verify that the onEvent method of the ServerSentEventListener is called multiple times.
    • Then: The HttpClient should establish an SSE connection and stream the response data to the listener.
    • Given: A proxy is configured.
    • When: Configure a proxy in the HttpClientBuilder and verify that the connection is established through the proxy.
    • Then: The HttpClient should use the proxy to connect to the OpenAI API.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: Provide an incorrect API key in the OpenAiClient builder.
    • Then: The HttpClient should return an appropriate error response, and the error handler should be invoked.
    • Given: The OpenAI API returns a non-200 status code.
    • When: Mock a non-200 status code in a test environment.
    • Then: The HttpClient should throw an exception, and the error handler should be invoked.
    • Given: The connection to the OpenAI API times out.
    • When: Set a very short connect timeout in the OpenAiClient builder and attempt to connect to a non-existent host.
    • Then: The HttpClient should throw a TimeoutException, and the error handler should be invoked.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests to the OpenAI API.
    • When: Simulate a high volume of requests using multiple threads.
    • Then: The HttpClient should handle the load without exhausting resources or causing timeouts.
    • Given: An OpenAI API endpoint that returns a very large SSE stream.
    • When: Mock a large SSE stream in a test environment.
    • Then: The SSE parser and listener should handle the stream efficiently without memory issues.
    • Given: Custom headers are provided to the OpenAI client.
    • When: Set custom headers in the OpenAiClient builder and verify they are present in the outgoing requests.
    • Then: The HttpClient should include these headers in all requests.

Documentation

Changes in this PR are sufficiently documented: ✅

@dliubarskyi dliubarskyi changed the title OpenAI: migrate to HttpClient [DRAFT] OpenAI: migrate to HttpClient Feb 11, 2025
@dliubarskyi dliubarskyi added this to the 1.0.0-beta2 milestone Feb 11, 2025
Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The removal of the proxy parameter from the builders of OpenAiChatModel, OpenAiEmbeddingModel, OpenAiImageModel, OpenAiLanguageModel, and OpenAiModerationModel might break existing code that relies on setting a proxy directly. Users now need to configure the proxy through the HttpClientBuilder.
  • The change in how timeouts are handled, now relying on HttpClientBuilder defaults if not explicitly set, could lead to unexpected timeout behavior if users were previously relying on the old default values.
  • The removal of the callTimeout parameter from the OpenAiClient.Builder might break existing code that relies on setting a call timeout directly. Users now need to configure the call timeout through the HttpClientBuilder.

Potential Design Issues

  • The increased reliance on HttpClientBuilder might make the configuration of the OpenAI models more complex, especially for users who are not familiar with the HttpClient library.
  • The internal implementation of the OpenAI client might not be as feature-rich or well-tested as the openai4j library, potentially leading to limitations or unexpected behavior.

Potential Bugs

  • If the HttpClientBuilder is not properly configured, the application might not be able to connect to the OpenAI API, leading to runtime errors.
  • If the new HttpClient implementation has different behavior regarding request retries or error handling, it could lead to unexpected behavior compared to the previous implementation.
  • The migration from openai4j to the internal implementation might introduce subtle differences in how requests are serialized or responses are deserialized, potentially leading to compatibility issues.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid OpenAI API key and a simple chat message.
    • When: Calling the generate() method on the OpenAiChatModel.
    • Then: The OpenAiChatModel should successfully generate a response from the OpenAI API.
    • Given: A streaming chat model and a long-running conversation.
    • When: Calling the generate() method on the OpenAiStreamingChatModel.
    • Then: The OpenAiStreamingChatModel should stream the response from the OpenAI API in chunks.
    • Given: A custom base URL for the OpenAI API.
    • When: Creating an OpenAiChatModel with a custom base URL.
    • Then: The OpenAiChatModel should use the custom base URL when making requests.

Suggested Negative Test Scenarios

    • Given: Invalid OpenAI API key.
    • When: Creating an OpenAiChatModel with an invalid API key and attempting to generate a response.
    • Then: The application should throw an appropriate exception (e.g., AuthenticationException) and not proceed with the request.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: Simulating a 500 error using WireMock.
    • Then: The HttpClient should retry the request up to the configured maximum number of retries.
    • Given: The OpenAI API is unavailable (e.g., network outage).
    • When: Simulating a network outage using WireMock.
    • Then: The HttpClient should throw a ConnectException or similar exception indicating the connection failure.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests to the OpenAI API.
    • When: Executing a performance test with multiple threads.
    • Then: The HttpClient should handle the load without exhausting resources or causing timeouts.
    • Given: An OpenAI API key with very limited quota.
    • When: Making several requests in quick succession.
    • Then: The application should gracefully handle the rate limiting errors and potentially retry after a delay.
    • Given: A custom HttpClientBuilder is provided with specific configurations (e.g., custom interceptors).
    • When: Building an OpenAiChatModel with a custom HttpClientBuilder.
    • Then: The OpenAiClient should use the provided HttpClientBuilder and respect its configurations.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The proxy parameter has been removed from the builders of OpenAiChatModel, OpenAiEmbeddingModel, OpenAiImageModel, OpenAiLanguageModel, and OpenAiModerationModel. Proxy configuration should now be done through the HttpClientBuilder.
  • The timeout parameter in the builders of OpenAiChatModel, OpenAiEmbeddingModel, OpenAiImageModel, OpenAiLanguageModel, and OpenAiModerationModel now represents both connect and read timeout. Separate connectTimeout and readTimeout are available via HttpClientBuilder.
  • The OpenAiHttpException has been replaced with HttpException. Consumers catching the former will need to update their code.
  • The default connect timeout has been changed from 60 seconds to 15 seconds.
  • The demo API key no longer works without explicitly setting the base URL to http://langchain4j.dev/demo/openai/v1.
  • The dependency on openai4j has been removed, and a dependency on langchain4j-http-client-jdk has been added. This may require consumers to update their dependency management.

Potential Design Issues

  • The removal of the proxy parameter from the builders might make it less convenient for consumers to configure proxy settings. They now need to use HttpClientBuilder directly.
  • The change in the timeout parameter's behavior (now representing both connect and read timeout) might be confusing for consumers who expect separate connect and read timeouts.
  • The migration to HttpClient introduces a new abstraction layer. While this offers flexibility, it also increases complexity and might make it harder to debug HTTP-related issues.

Potential Bugs

  • The streaming might not be closed even if there is no [DONE] event sent from the LLM provider API. This is explicitly mentioned in the TODOs, so it's a known potential bug.
  • If the HttpClient is not properly configured, it could lead to unexpected behavior, such as connection leaks or incorrect proxy settings.
  • If custom headers are not handled correctly, it could lead to authentication or authorization issues.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: A streaming request is made to the OpenAI API.
    • Then: The OpenAiChatModel should successfully generate a response, and the onNext method of the StreamingResponseHandler should be called for each token in the response.
    • Given: A valid API key and prompt are provided, and the logRequests and logResponses flags are set to true.
    • When: A request is made to the OpenAI API with logging enabled.
    • Then: The HTTP requests and responses should be logged to the console.
    • Given: A valid API key and prompt are provided, and a custom HttpClientBuilder is provided.
    • When: The OpenAiChatModel is built with a custom HttpClientBuilder.
    • Then: The OpenAiChatModel should use the provided HttpClientBuilder to create the HttpClient instance.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: The OpenAiChatModel is initialized with an incorrect API key.
    • Then: An HttpException should be thrown with a 401 status code.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: The OpenAI API experiences a server-side error.
    • Then: An HttpException should be thrown, and the request should be retried up to maxRetries times. If all retries fail, the last exception should be propagated.
    • Given: The connection to the OpenAI API times out.
    • When: The connection or read timeout is exceeded.
    • Then: An HttpException should be thrown, and the request should be retried.

Suggested Corner Case Test Scenarios

    • Given: A streaming response is interrupted mid-stream due to a network error.
    • When: The connection to the OpenAI API is severed during a streaming request.
    • Then: The onError method of the StreamingResponseHandler should be called with an appropriate exception, and the stream should be gracefully closed.
    • Given: The OpenAI API returns a malformed JSON response.
    • When: The OpenAI API returns invalid JSON.
    • Then: The onError method of the StreamingResponseHandler should be called with a JsonProcessingException or similar, and the stream should be closed.
    • Given: A custom HttpClientBuilder is provided with specific configurations (e.g., custom interceptors).
    • When: The OpenAiChatModel is built with a custom HttpClientBuilder.
    • Then: The OpenAiChatModel should use the provided HttpClientBuilder to create the HttpClient instance, and the custom configurations should be applied.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The migration from openai4j to HttpClient introduces breaking changes related to exception handling. OpenAiHttpException is replaced with HttpException. Consumers relying on catching OpenAiHttpException will need to update their code.
  • The default connect timeout has been changed from 60 seconds to 15 seconds. Consumers relying on the previous default timeout may experience connection timeouts if their network conditions require a longer timeout.
  • The proxy configuration is removed from the builder. Consumers using proxy settings will need to configure the HttpClientBuilder directly.
  • The demo API key no longer works without explicitly setting the base URL. Consumers using the demo key will need to update their code to include the base URL.
  • The timeout parameter is deprecated in favor of connectTimeout and readTimeout. Consumers using timeout will need to migrate to the new parameters.

Potential Design Issues

  • The introduction of SyncOrAsyncOrStreaming and related interfaces adds complexity to the API. It might be beneficial to provide clearer documentation and examples to guide users on how to use these new interfaces effectively.
  • The dependency on Jackson for JSON serialization/deserialization introduces a potential design issue if other parts of the project rely on a different JSON library. This could lead to conflicts or inconsistencies.
  • The use of ServiceLoader for OpenAiClientBuilderFactory and HttpClientBuilderLoader might introduce issues with classloading in certain environments (e.g., OSGi).

Potential Bugs

  • The streaming implementation might not be as robust as the previous one, potentially leading to issues with long-running streams or error handling.
  • The new HttpClient might have different behavior regarding header handling, potentially causing issues with authentication or other custom headers.
  • The removal of the proxy parameter might introduce issues for users relying on proxy configurations, especially if the HttpClientBuilder is not configured correctly.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid prompt is sent to the OpenAI API.
    • When: The generate method is called with a valid prompt and API key.
    • Then: The HttpClient should successfully send the request and receive a valid response, which is then parsed and returned.
    • Given: The logRequests and logResponses flags are set to true.
    • When: The OpenAiClient is created with logRequests and logResponses enabled.
    • Then: The LoggingHttpClient should log the request and response details to the configured logger.
    • Given: Custom headers are provided to the OpenAiClient.
    • When: The OpenAiClient is created with a customHeaders map.
    • Then: The HttpClient should include the custom headers in the request.

Suggested Negative Test Scenarios

    • Given: The OpenAI API returns a 429 (Too Many Requests) error.
    • When: The OpenAI API is rate-limited.
    • Then: The retry mechanism should be triggered, and the request should be retried up to the configured maxRetries times. After exceeding the retries, an HttpException should be thrown.
    • Given: The API key is invalid or missing.
    • When: The apiKey is not set or is incorrect.
    • Then: The OpenAiClient should throw an HttpException with a 401 (Unauthorized) status code.
    • Given: The network connection is unstable, causing intermittent connection resets.
    • When: The execute method encounters a SocketException or similar network-related exception.
    • Then: The HttpClient should handle connection resets gracefully, and the request should be retried if appropriate.

Suggested Corner Case Test Scenarios

    • Given: A very large prompt is sent to the OpenAI API.
    • When: The generate method is called with a prompt exceeding typical size limits.
    • Then: The HttpClient should handle the large request size without errors, and the response should be processed correctly.
    • Given: The OpenAI API returns a malformed JSON response.
    • When: The execute method receives a response that cannot be parsed as the expected class.
    • Then: The Json.fromJson method should throw a RuntimeException, and the error handler should be invoked.
    • Given: A custom HttpClientBuilder is provided with specific configurations (e.g., interceptors, SSL settings).
    • When: The OpenAiClient.builder() method is called with a custom HttpClientBuilder.
    • Then: The OpenAiClient should use the provided HttpClientBuilder to create the HttpClient, and the custom configurations should be applied.

Documentation

Changes in this PR are sufficiently documented: ✅

Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The OpenAiHttpException is replaced with HttpException. Consumers catching OpenAiHttpException will need to update their code.
  • The default connect timeout is changed from 60 seconds to 15 seconds. Consumers relying on the previous default may experience connection timeouts.
  • The proxy parameter is removed from the builders. Consumers using this parameter will need to configure the HttpClientBuilder directly.
  • The persistTo and withPersisting parameters are removed from the OpenAiImageModelBuilder. Consumers using these parameters will need to implement their own persistence logic.
  • The demo API key is removed. Consumers using this key will need to set the baseUrl manually.
  • The dev.ai4j logging level is removed. Consumers relying on this logging level will need to use dev.langchain4j instead.

Potential Design Issues

  • The removal of the proxy parameter from the builders might make it less convenient for users to configure a proxy. They now need to use HttpClientBuilder directly.
  • The introduction of HttpClientBuilder might increase the complexity of configuring the OpenAI models.

Potential Bugs

  • The migration to HttpClient might introduce new bugs related to HTTP request handling, especially around timeouts, connection pooling, and error handling.
  • The streaming implementation might have issues with closing the stream correctly if the LLM provider does not send a [DONE] event.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: Calling the generate method on a language model.
    • Then: The model should successfully generate a completion and return the result.
    • Given: A valid API key and a list of chat messages are provided.
    • When: Calling the chat method on a chat model.
    • Then: The model should successfully generate a chat completion and return the result.
    • Given: A valid API key and text segments are provided.
    • When: Calling the embedAll method on an embedding model.
    • Then: The model should successfully generate embeddings for the text segments and return the results.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: Creating an OpenAi model with an invalid API key.
    • Then: The model should return an appropriate error indicating authentication failure.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: Making a request to the OpenAI API.
    • Then: The retry mechanism should be triggered, and if retries fail, an appropriate exception should be thrown.
    • Given: The base URL is malformed or unreachable.
    • When: Creating an OpenAi model with a malformed base URL.
    • Then: The model should throw an exception indicating a connection error.

Suggested Corner Case Test Scenarios

    • Given: A very large input text that exceeds the token limit.
    • When: Generating embeddings or completions with a large text.
    • Then: The model should handle the truncation gracefully and return a partial or error response as configured.
    • Given: A network connection that is intermittently unstable.
    • When: Executing a long-running streaming request.
    • Then: The retry mechanism should handle temporary network issues and successfully complete the request.
    • Given: Custom headers are provided with conflicting values to the default headers.
    • When: Building an OpenAi model with custom headers.
    • Then: The custom headers should override the default headers.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 11, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration to HttpClient introduces breaking changes related to proxy configuration, exception handling (OpenAiHttpException -> HttpException), default timeouts (connect 60->15), and the removal of the demo key. Consumers relying on the old proxy configuration, exception types, or the demo key will need to update their code.
  • The removal of the persistTo and with-persisting options from the image model configuration.
  • The removal of logStreamingResponses from the OpenAiClient builder.
  • The removal of callTimeout from the OpenAiClient builder, replaced by connectTimeout and readTimeout.

Potential Design Issues

  • The migration to HttpClient might introduce unnecessary complexity or overhead compared to the previous implementation.
  • The new builder pattern for configuring the OpenAI models might be less intuitive or flexible than the previous approach.
  • The introduction of separate connectTimeout and readTimeout might be confusing for users who are accustomed to a single timeout value.

Potential Bugs

  • The new HttpClient might not handle all edge cases related to streaming responses correctly, potentially leading to incomplete or corrupted data.
  • The retry mechanism might not be robust enough to handle all types of transient errors, potentially leading to request failures.
  • There might be inconsistencies in how token usage is calculated with the new HttpClient, leading to inaccurate billing or rate limiting.
  • The new implementation might not properly close the streaming connection in all cases, potentially leading to resource leaks.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and a simple prompt are provided.
    • When: Creating an OpenAiChatModel with a valid API key and calling the generate method with a simple prompt.
    • Then: The model should return a relevant and coherent response.
    • Given: Custom headers are provided for the API request.
    • When: Creating an OpenAiChatModel with custom headers and verifying that the headers are present in the request.
    • Then: The custom headers should be included in the API request.
    • Given: A timeout is specified for the API request.
    • When: Creating an OpenAiChatModel with a specific timeout and verifying that the request completes within that time.
    • Then: The API request should be completed within the specified timeout period.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: Creating an OpenAiChatModel with an incorrect or expired API key.
    • Then: The model should return an appropriate error message indicating that the API key is invalid.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: The OpenAI API experiences a temporary server-side issue.
    • Then: The retry mechanism should be triggered, and after exceeding the maximum number of retries, an exception should be thrown.
    • Given: An empty prompt is sent to the language model.
    • When: Calling the generate method with an empty or null prompt string.
    • Then: The model should return an error or a default response, rather than crashing or producing unexpected output.

Suggested Corner Case Test Scenarios

    • Given: A very large input text that exceeds the maximum token limit.
    • When: Generating embeddings or completions with an input text that is significantly larger than the model's maximum token limit.
    • Then: The model should handle the truncation gracefully and return a valid response, potentially indicating that the input was truncated.
    • Given: A network connection that is intermittently unstable.
    • When: Executing requests with a network connection that experiences occasional drops or slowdowns.
    • Then: The retry mechanism should handle temporary network issues and successfully complete the request after a few retries.
    • Given: Custom headers that conflict with internally set headers.
    • When: Providing custom headers that have the same keys as the default headers set by the client.
    • Then: The custom headers should override the default headers, ensuring that the user-specified configuration takes precedence.

Documentation

Changes in this PR are sufficiently documented: ❌

Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration from openai4j to a custom HttpClient implementation introduces several breaking changes:
  1. OpenAiHttpException is replaced with HttpException.
  2. Default timeouts have changed (connect timeout from 60s to 15s).
  3. The demo API key no longer works without explicitly setting the base URL.
  4. Proxy configuration properties have been removed (e.g., langchain4j.open-ai.chat-model.proxy.host).
  5. The persistTo and withPersisting properties for image models have been removed.
  • The dev.ai4j.openai4j logging level is removed, requiring users to adjust their logging configurations.
  • The apiKey is now not mandatory. This might break existing code that relies on the apiKey being present.

Potential Design Issues

  • The introduction of a custom HttpClient implementation might lead to inconsistencies in HTTP client behavior compared to the previous openai4j library.
  • The removal of the demo API key without a clear migration path might inconvenience users relying on the demo key for testing.
  • The migration introduces a dependency on langchain4j-http-client-jdk. It's important to ensure that this dependency doesn't introduce conflicts or performance regressions.

Potential Bugs

  • ⚠️ The TODO list includes ensuring that streaming is closed even if there is no [DONE] event. Failure to do so could lead to resource leaks.
  • The removal of proxy configuration properties might lead to issues for users relying on proxy settings.
  • The change in default timeouts (connect 60s -> 15s) might cause issues for users with slow network connections.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: The generate method is called with a valid prompt.
    • Then: The OpenAiChatModel should successfully generate a response, and the onNext and onComplete handlers should be called in streaming scenarios.
    • Given: Custom headers are provided.
    • When: The OpenAiChatModelBuilder is configured with custom headers.
    • Then: The HttpClient should include the custom headers in the request to the OpenAI API.
    • Given: Logging is enabled.
    • When: The logRequests and logResponses flags are set to true.
    • Then: Requests and responses should be logged according to the configured logging level.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: The OpenAiChatModel is initialized with an incorrect API key.
    • Then: An HttpException should be thrown, and the onError handler should be called in streaming scenarios.
    • Given: The OpenAI API is unavailable or returns a 500 error.
    • When: The execute method fails due to a server error.
    • Then: An HttpException should be thrown, and the retry mechanism should be triggered. If retries fail, the exception should be propagated.
    • Given: The connection timeout is set to a very short duration.
    • When: The connectTimeout is set to a value less than the time required to establish a connection.
    • Then: An HttpException due to timeout should be thrown.

Suggested Corner Case Test Scenarios

    • Given: A very large prompt is sent to the OpenAI API.
    • When: The generate method is called with a prompt exceeding typical size limits.
    • Then: The HttpClient should handle the large request size without errors, and the response should be processed correctly.
    • Given: The OpenAI API returns a malformed JSON response.
    • When: The execute method receives a response that cannot be parsed as a ChatCompletionResponse.
    • Then: The JSON parsing logic should gracefully handle the error and propagate it to the user, potentially through the onError handler in streaming scenarios.
    • Given: A custom HttpClientBuilder is provided with specific configurations (e.g., interceptors, SSL settings).
    • When: The OpenAiChatModelBuilder is configured with a custom HttpClientBuilder.
    • Then: The OpenAiChatModel should correctly use the provided HttpClientBuilder to create the HttpClient.

Documentation

Changes in this PR are sufficiently documented: ❌

Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration to HttpClient introduces breaking changes related to proxy configuration, timeouts, exception handling, and the removal of the demo key. Consumers relying on the previous configuration properties (langchain4j.open-ai..proxy.) will need to adapt to the new HttpClient configuration. The default connect timeout has been reduced from 60 seconds to 15 seconds, which may affect applications in environments with slow network connections. OpenAiHttpException is replaced with HttpException, requiring changes in exception handling logic. The demo key is removed, requiring users to set baseUrl manually.
  • The removal of the persistTo and with-persisting properties from the OpenAiImageModel configuration.
  • The removal of logStreamingResponses from the OpenAiClient builder, replaced with logResponses

Potential Design Issues

  • The introduction of DefaultOpenAiClient and the associated builder pattern might add complexity if not carefully managed. Consider whether the builder pattern is truly necessary for all configuration options.
  • The migration introduces a dependency on langchain4j-http-client-jdk. Consider the implications of forcing a specific HTTP client implementation on users.
  • The removal of proxy settings from the configuration and reliance on the HttpClientBuilder might make proxy configuration less straightforward for some users.

Potential Bugs

  • The TODO in HttpClient.java regarding the default parser suggests that the default SSE parser might not be fully implemented or tested, potentially leading to issues with SSE event handling.
  • The PR description mentions ensuring that streaming is closed even if there is no [DONE] event. Failure to properly close the stream could lead to resource leaks.
  • The removal of the dependency conflict resolution for openai4j might reintroduce version conflicts if the underlying dependencies are not properly managed.
  • ⚠️ The LocalAi*IT tests are failing because the base URL and model names are hardcoded and incorrect.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and a well-formed request are provided.
    • When: Calling the OpenAI API with valid credentials and a correct request.
    • Then: The HttpClient should successfully execute the request and return the expected response from the OpenAI API.
    • Given: Streaming is enabled, and the OpenAI API returns a stream of SSE events.
    • When: Receiving a stream of SSE events from the OpenAI API.
    • Then: The ServerSentEventParser should correctly parse each event and deliver it to the ServerSentEventListener.
    • Given: Custom headers are provided in the request.
    • When: Adding custom headers to the request.
    • Then: The HttpClient should include these headers in the outgoing HTTP request.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: Calling the OpenAI API with an incorrect API key.
    • Then: An appropriate HttpException should be thrown, indicating authentication failure.
    • Given: The OpenAI API returns a 429 (Too Many Requests) error.
    • When: Exceeding the rate limit of the OpenAI API.
    • Then: The HttpClient should retry the request after a suitable delay, as determined by the retry policy.
    • Given: The OpenAI API returns an unexpected error code (e.g., 500).
    • When: Receiving an unexpected error code from the OpenAI API.
    • Then: An appropriate HttpException should be thrown, indicating the server-side error.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: Executing multiple requests in parallel.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: The OpenAI API returns a malformed SSE event.
    • When: Receiving a malformed SSE event.
    • Then: The ServerSentEventParser should handle the error gracefully and continue processing subsequent events.
    • Given: The network connection is unstable and prone to intermittent failures.
    • When: Experiencing network connectivity issues.
    • Then: The HttpClient should retry failed requests according to the configured retry policy.

Documentation

Changes in this PR are sufficiently documented: ❌

WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration to HttpClient introduces several breaking changes, including the removal of the proxy parameter, changes to exception types (OpenAiHttpException -> HttpException), and changes to default timeouts (connect 60->15). The removal of the demo key also requires users to manually set the baseUrl.
  • The dev.ai4j logging level is removed, which might affect users who rely on it for debugging.
  • The persistTo and withPersisting parameters are removed from OpenAiImageModel, which might affect users who rely on local image persistence.

Potential Design Issues

  • The migration to HttpClient introduces a dependency on langchain4j-http-client-jdk in langchain4j-open-ai. This might increase the overall complexity of the project.
  • The removal of the proxy parameter might limit the flexibility of the OpenAiClient in certain deployment scenarios.
  • The new DefaultOpenAiClient is not configurable via SPI. This might limit the ability to customize the client's behavior in certain scenarios.

Potential Bugs

  • The new HttpClient might have subtle differences in behavior compared to the previous implementation, potentially leading to unexpected issues with request handling, error handling, or streaming.
  • The DefaultServerSentEventParser might not correctly handle all possible SSE event formats, potentially leading to data loss or parsing errors.
  • The removal of the demo key and the requirement to set the baseUrl manually might lead to configuration errors for new users.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and a well-formed request.
    • When: Calling any of the OpenAI API methods with valid credentials and input.
    • Then: The OpenAiClient should successfully call the OpenAI API and return the expected response.
    • Given: A streaming request is made to the OpenAI API.
    • When: Calling the chatCompletion or completion methods with stream = true.
    • Then: The DefaultServerSentEventParser should correctly parse the SSE stream and deliver the parsed events to the ServerSentEventListener.
    • Given: Custom headers are provided to the OpenAiClient.
    • When: Configuring the OpenAiClient with a customHeaders map.
    • Then: The HttpClient should include these headers in the HTTP requests it sends to the OpenAI API.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: Calling any of the OpenAI API methods with an incorrect API key.
    • Then: The OpenAiClient should return an appropriate error response (e.g., 401 Unauthorized) and the client should handle the error gracefully, potentially retrying the request or notifying the user.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: Simulating a temporary server-side error on the OpenAI API.
    • Then: The client should retry the request up to the configured maximum number of retries. After exceeding the maximum retries, it should throw an appropriate exception.
    • Given: The network connection is interrupted during an SSE stream.
    • When: Disconnecting the network connection while streaming data from the OpenAI API.
    • Then: The DefaultServerSentEventParser should handle the IOException and notify the ServerSentEventListener of the error.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests to the OpenAI API.
    • When: Executing a performance test with multiple threads sending requests simultaneously.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: An SSE stream that contains malformed events or unexpected data formats.
    • When: Simulating a flaky network connection or a server that occasionally sends invalid SSE data.
    • Then: The DefaultServerSentEventParser should handle these gracefully, either by skipping the malformed events or by providing a meaningful error message to the ServerSentEventListener.
    • Given: A request that results in a very large response body.
    • When: Requesting a large embedding or a very long completion.
    • Then: The HttpClient should be able to handle the large response without running out of memory.

Documentation

Changes in this PR are sufficiently documented: ❌

WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration to HttpClient introduces breaking changes related to proxy configuration, exception handling (OpenAiHttpException -> HttpException), default timeouts (connect 60->15), and the removal of the demo key. Consumers relying on the old proxy settings or exception types will need to update their code.
  • The default connect timeout has been reduced from 60 seconds to 15 seconds. This might cause issues for users with slow network connections.
  • The persistTo and withPersisting parameters have been removed from the OpenAiImageModel. This will break code that relies on automatic image persistence.
  • ⚠️ The proxy parameter has been removed from all model builders. Users now need to configure the proxy via the HttpClientBuilder. This is a significant change in how proxy settings are handled.
  • The logStreamingResponses parameter in LocalAiStreamingChatModel and LocalAiStreamingLanguageModel has been replaced with logResponses. This might affect users who were specifically logging streaming responses.

Potential Design Issues

  • The migration to HttpClient introduces a new dependency on langchain4j-http-client-jdk. This increases the overall complexity of the project.
  • The changes in LocalAI models, where the OpenAiClient is built with an ignored API key and different timeout handling, might indicate a divergence in design between the OpenAI and LocalAI implementations.
  • The removal of individual proxy parameters from model builders and the introduction of httpClientBuilder might make the configuration more complex for simple use cases.

Potential Bugs

  • ⚠️ The PR description mentions ensuring that streaming is closed even if there is no [DONE] event. Failure to do so could lead to resource leaks.
  • The removal of the demo key and the need to set the baseUrl manually might lead to confusion and errors for new users.
  • The changes in LocalAI models where openAiApiKey is ignored and timeouts are handled differently might introduce inconsistencies or unexpected behavior compared to the OpenAI models.
  • The test changes in LocalAI models show that tokenUsage is sometimes null and sometimes not null. This inconsistency needs to be investigated.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided
    • When: The generate method is called with a valid prompt.
    • Then: The OpenAI API should return a successful response, and the answer should be parsed correctly.
    • Given: A proxy is configured
    • When: The proxy parameter is set to a valid proxy configuration.
    • Then: The HttpClient should route requests through the proxy server.
    • Given: Custom headers are provided
    • When: The customHeaders parameter is set to a map of headers.
    • Then: The HttpClient should include the custom headers in the request.

Suggested Negative Test Scenarios

    • Given: Invalid API key is used
    • When: The apiKey parameter is set to an empty string or null.
    • Then: An appropriate exception (e.g., HttpException) should be thrown, indicating authentication failure.
    • Given: The OpenAI API returns a 429 (Too Many Requests) error
    • When: The rate limit is exceeded.
    • Then: The client should retry the request up to maxRetries times, with exponential backoff.
    • Given: The OpenAI API is unavailable
    • When: The OpenAI API server is down or unreachable.
    • Then: The client should retry the request up to maxRetries times, with exponential backoff, and eventually throw an exception if the API remains unavailable.

Suggested Corner Case Test Scenarios

    • Given: A very large prompt is sent to the OpenAI API
    • When: The maxRetries parameter is set to a high value, and the initial requests fail due to network issues.
    • Then: The HttpClient should handle the large payload without errors, and the connection should not timeout prematurely.
    • Given: The OpenAI API returns a malformed JSON response
    • When: The connection to the OpenAI API is unstable, causing intermittent network errors.
    • Then: The JSON parsing should fail gracefully, and the onError handler should be invoked.

Documentation

Changes in this PR are sufficiently documented: ❌

WIP
WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • ⚠️ The migration from openai4j to a custom HttpClient implementation introduces several breaking changes. OpenAiHttpException is replaced with HttpException. Default timeouts have changed (connect timeout from 60s to 15s). The demo key now requires setting the base URL manually. The proxy configuration is removed.
  • The persistTo and withPersisting parameters have been removed from the OpenAiImageModel. This will break any code that relies on the automatic persistence of generated images.
  • The apiKey is now mandatory. Previously, it was possible to use the OpenAI models without an API key when using a custom base URL. This is no longer possible.
  • The dev.ai4j.openai4j logging level has been removed. Any code that relies on this logging level will need to be updated.

Potential Design Issues

  • The new HttpClient implementation might not be as performant as the previous openai4j implementation, especially for streaming responses.
  • The increased complexity of the OpenAiClient builder might make it more difficult to configure the model correctly.

Potential Bugs

  • The new HttpClient implementation might have bugs related to handling different HTTP status codes, especially error codes. The previous implementation might have handled these cases more robustly.
  • The SSE parsing logic in DefaultServerSentEventParser might have edge cases that are not covered by the existing tests, potentially leading to incorrect parsing of streamed responses.
  • The removal of the proxy configuration might cause issues for users who rely on a proxy server to access the OpenAI API.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: Calling the generate method with a simple prompt.
    • Then: The model should generate a relevant and coherent response.
    • Given: A tool specification is provided.
    • When: Using the model with a tool that performs a simple calculation.
    • Then: The model should correctly identify and execute the tool, incorporating the results into its response.
    • Given: Streaming is enabled.
    • When: Calling the generate method with streaming enabled.
    • Then: The model should stream the response in chunks, and the StreamingResponseHandler should receive each chunk.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: Initializing the OpenAiChatModel with an incorrect API key.
    • Then: The model should throw an appropriate authentication exception.
    • Given: The OpenAI API returns a 429 (Rate Limit Exceeded) error.
    • When: Making multiple requests to the OpenAI API in a short period, exceeding the rate limit.
    • Then: The retry mechanism should engage, and after exceeding the maximum retries, an exception should be thrown.
    • Given: The network connection is unstable, causing intermittent read timeouts.
    • When: Making a request to the OpenAI API over a flaky network connection.
    • Then: The HTTP client should handle the timeouts and retry the request up to the maximum number of retries.

Suggested Corner Case Test Scenarios

    • Given: A very large input string that approaches the maximum allowed token limit.
    • When: Generating text with a model using the maximum allowed input tokens.
    • Then: The tokenizer should accurately count the tokens, and the model should handle the request without errors related to token limits.
    • Given: Custom HTTP headers are set with unusual characters or encodings.
    • When: Making a request to the OpenAI API with custom headers containing special characters.
    • Then: The HTTP client should correctly encode and transmit these headers without causing errors in the request or response.
    • Given: The OpenAI API returns a malformed SSE event (e.g., missing 'data:' prefix, invalid JSON).
    • When: Streaming data from OpenAI with a corrupted SSE stream.
    • Then: The SSE parser should gracefully handle the error and either skip the event or propagate an appropriate exception to the error handler.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The removal of dev.ai4j.openai4j.OpenAiHttpException and its replacement with dev.langchain4j.exception.HttpException is a breaking change for anyone catching the former exception.
  • The removal of default modelName for OpenAiEmbeddingModel, OpenAiLanguageModel, and OpenAiStreamingLanguageModel requires users to explicitly set these values, potentially breaking existing code.
  • The removal of withPersisting and persistTo properties from OpenAiImageModel removes previously supported functionality.
  • The removal of langchain4j.open-ai.*-model.proxy.* Spring Boot properties requires users to configure proxy settings directly on the HTTP client, potentially breaking existing configurations.
  • The change in default connect timeout from 60 seconds to 15 seconds may cause issues for users with slow network connections.
  • The apiKey is now optional for all OpenAi*Models. This might break existing code that relies on the apiKey being mandatory.

Potential Design Issues

  • The increased complexity of configuring the HTTP client might make the library harder to use for some users.
  • The removal of dedicated proxy properties might make proxy configuration less discoverable for users relying on Spring Boot's automatic configuration.
  • The introduction of HttpClient SPI might lead to fragmentation and compatibility issues if different HTTP client implementations are not thoroughly tested and maintained.

Potential Bugs

  • The migration to HttpClient might introduce new bugs related to HTTP request handling, especially around error handling, timeouts, and SSE parsing.
  • The removal of default model names and temperature values might lead to unexpected behavior if users forget to set them explicitly.
  • The new HTTP client configuration might not be as robust as the previous OkHttp/Retrofit implementation in handling various network conditions and edge cases.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: Calling the generate method on an OpenAiChatModel instance.
    • Then: The OpenAiChatModel should generate a response successfully.
    • Given: A proxy is configured on the HttpClient.
    • When: The OpenAi*Model is instantiated with a HttpClientBuilder that has a proxy configured.
    • Then: All requests to the OpenAI API should be routed through the configured proxy.
    • Given: SSE streaming is enabled.
    • When: Using OpenAiStreamingChatModel.
    • Then: The response should be streamed back to the client via Server-Sent Events.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: The OpenAi*Model is instantiated with an incorrect API key.
    • Then: An HttpException should be thrown with an appropriate error message.
    • Given: The OpenAI API is unavailable.
    • When: The OpenAI API server is down or unreachable.
    • Then: An HttpException should be thrown after the configured number of retries.
    • Given: The user provides invalid JSON schema for structured outputs.
    • When: Using JSON schema with OpenAiChatModel.
    • Then: An exception should be thrown during the API call, indicating the schema is invalid.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: The application is under heavy load.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: The OpenAI API returns a malformed SSE response.
    • When: The OpenAI API returns a corrupted SSE stream.
    • Then: The DefaultServerSentEventParser should handle the error gracefully and invoke the onError method of the ServerSentEventListener.
    • Given: A custom HttpClient implementation is provided.
    • When: The OpenAi*Model is instantiated with a custom HttpClientBuilder.
    • Then: The application should use the custom HttpClient for all OpenAI API requests.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • dev.ai4j.openai4j.OpenAiHttpException is now replaced by dev.langchain4j.exception.HttpException
  • The default connect timeout is now 15 seconds instead of 60 for all OpenAi*Models
  • To continue using the "demo" key, you need to specify the baseUrl explicitly.
  • If you are using Proxy, you now need to configure it directly on the HTTP client of your choice. All langchain4j.open-ai.*-model.proxy.* Spring Boot properties are not working any more.
  • OpenAiEmbeddingModel: removed default modelName (TEXT_EMBEDDING_ADA_002), please set it explicitly now
  • OpenAiImageModel: removed withPersisting and persistTo properties. This functionality will not be supported any more.
  • OpenAiLanguageModel and OpenAiStreamingLanguageModel: removed default modelName (GPT_3_5_TURBO_INSTRUCT) and temperature (0.7), please set it explicitly now
  • OpenAiModerationModel: removed default modelName (TEXT_MODERATION_LATEST), please set it explicitly now
  • apiKey is now optional for all OpenAi*Models

Potential Design Issues

  • The increased flexibility in configuring the HTTP client comes at the cost of increased complexity for users who simply want to use the default configuration.
  • The removal of default values for model names and temperature might make the API less user-friendly for new users.
  • The need to explicitly set the baseUrl when using the 'demo' key adds friction to the demonstration process.

Potential Bugs

  • The migration to HttpClient might introduce subtle differences in how HTTP requests are handled, potentially leading to compatibility issues with certain OpenAI API features or edge cases.
  • The removal of default values for model names and temperature could lead to unexpected behavior if users forget to set them explicitly.
  • The new proxy configuration might not be as straightforward as the previous property-based approach, potentially leading to configuration errors.
  • The removal of withPersisting and persistTo properties in OpenAiImageModel might break existing workflows that rely on this functionality.

Testing

Changes in this PR are sufficiently tested: ❌

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt.
    • When: Using any of the OpenAI models.
    • Then: The model should return a valid response.
    • Given: A custom HttpClient is provided.
    • When: Using any of the OpenAI models.
    • Then: The OpenAI client should use the custom HttpClient.
    • Given: A custom HttpClientBuilder is provided.
    • When: Using any of the OpenAI models.
    • Then: The OpenAI client should use the custom HttpClient built by the builder.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: Using any of the OpenAI models.
    • Then: An appropriate HttpException should be thrown.
    • Given: The OpenAI API returns a 500 error.
    • When: Using any of the OpenAI models.
    • Then: An appropriate HttpException should be thrown, potentially after retries.
    • Given: The OpenAI API returns a malformed JSON response.
    • When: Using any of the OpenAI models.
    • Then: An exception should be thrown during JSON parsing.

Suggested Corner Case Test Scenarios

    • Given: A custom HttpClient implementation is provided that throws an exception after a few successful requests.
    • When: Using any of the OpenAI models.
    • Then: The retry mechanism should handle this and continue until maxRetries is reached, then propagate the exception.
    • Given: A very slow network connection.
    • When: Using any of the OpenAI models.
    • Then: The connectTimeout and readTimeout settings should be respected, and the appropriate exceptions thrown.
    • Given: A custom HttpClient implementation is provided that modifies the request headers.
    • When: Using any of the OpenAI models.
    • Then: The OpenAI client should use the modified headers.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 12, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The removal of dev.ai4j.openai4j.OpenAiHttpException and its replacement with dev.langchain4j.exception.HttpException is a breaking change for code that directly catches the former exception.
  • The removal of default modelName for OpenAiEmbeddingModel, OpenAiLanguageModel, and OpenAiStreamingLanguageModel requires users to explicitly set these values, potentially breaking existing configurations.
  • The removal of withPersisting and persistTo properties from OpenAiImageModel removes previously supported functionality.
  • The change in default connect timeout from 60 seconds to 15 seconds may cause issues for users with slow network connections.
  • The removal of langchain4j.open-ai.*-model.proxy.* Spring Boot properties requires users to configure proxies directly on the HTTP client, potentially breaking existing configurations.
  • The apiKey is now optional for all OpenAi*Models, which might change the behavior if the user was relying on the previous required state.

Potential Design Issues

  • The increased flexibility in configuring the HTTP client might add complexity to the API.
  • The removal of default values might make the API less user-friendly for new users.

Potential Bugs

  • The migration to HttpClient might introduce bugs related to request handling, response parsing, or error handling.
  • The removal of default values for certain parameters might lead to unexpected behavior if users are relying on those defaults.
  • The new proxy configuration method might not be as straightforward as the previous Spring Boot properties, potentially leading to configuration errors.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key is used.
    • When: A request is made to the OpenAI API.
    • Then: The OpenAI API should return a successful response.
    • Given: A proxy is configured on the HttpClient.
    • When: A request is made to the OpenAI API.
    • Then: The HttpClient should route requests through the proxy.
    • Given: Logging is enabled.
    • When: A request is made to the OpenAI API.
    • Then: Requests and responses should be logged.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: The OpenAI API returns an authentication error.
    • Then: An appropriate HttpException should be thrown.
    • Given: The OpenAI API returns a rate limit error.
    • When: The OpenAI API returns a rate limit error.
    • Then: The HttpClient should retry the request according to the configured retry policy.
    • Given: The OpenAI API is unavailable.
    • When: The OpenAI API is unavailable.
    • Then: An appropriate HttpException should be thrown.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: The application is under heavy load.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: The OpenAI API returns a malformed SSE response.
    • When: The OpenAI API returns a malformed SSE response.
    • Then: The SSE parser should handle the error gracefully and invoke the onError method of the ServerSentEventListener.
    • Given: A custom HttpClient implementation is provided.
    • When: A custom HttpClient implementation is provided.
    • Then: The application should use the custom HttpClient implementation instead of the default one.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 13, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • Removal of dev.ai4j.openai4j.OpenAiHttpException and replacement with dev.langchain4j.exception.HttpException.
  • Removal of default modelName for OpenAiEmbeddingModel (TEXT_EMBEDDING_ADA_002), requiring explicit setting.
  • Removal of withPersisting and persistTo properties from OpenAiImageModel.
  • Removal of default modelName (GPT_3_5_TURBO_INSTRUCT) and temperature (0.7) from OpenAiLanguageModel and OpenAiStreamingLanguageModel, requiring explicit setting.
  • Removal of default modelName (TEXT_MODERATION_LATEST) from OpenAiModerationModel, requiring explicit setting.
  • All OpenAi*Model constructors now accept a builder object instead of individual properties.
  • The langchain4j.open-ai.*-model.proxy.* Spring Boot properties are no longer working.
  • The default connect timeout is now 15 seconds instead of 60 for all OpenAi*Models.

Potential Design Issues

  • The migration to java.net.http.HttpClient introduces a dependency on a newer Java version. This might limit the adoption of the library in projects using older Java versions.
  • The removal of the withPersisting and persistTo properties from OpenAiImageModel reduces the functionality of the image generation feature. Users who relied on this feature will need to find alternative solutions for persisting generated images.

Potential Bugs

  • Potential issues with SSE parsing in DefaultServerSentEventParser, especially with complex or malformed events. The TODO comments in the code suggest this area needs more review and testing.
  • The removal of default values for model names and temperature might lead to unexpected behavior if users forget to set them explicitly. This could result in errors or the use of unintended models.
  • The change to HttpClient might introduce compatibility issues with existing proxy configurations, as users now need to configure the proxy directly on the HTTP client.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and model name are provided.
    • When: The chat method is called with a user message.
    • Then: The OpenAiChatModel should successfully generate a response.
    • Given: A custom HttpClient is configured with a proxy.
    • When: The httpClientBuilder is configured with proxy settings.
    • Then: The OpenAiChatModel should use the proxy when making requests to the OpenAI API.
    • Given: Logging is enabled.
    • When: The logRequests and logResponses flags are set to true.
    • Then: Requests and responses should be logged to the console or configured logging output.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: The OpenAi*Model is instantiated with an incorrect API key.
    • Then: An HttpException should be thrown, indicating authentication failure.
    • Given: The OpenAI API is unavailable.
    • When: There is a network outage or the OpenAI API is experiencing downtime.
    • Then: An HttpException should be thrown, indicating a connection or timeout issue.
    • Given: The user attempts to use the 'demo' key without specifying the base URL.
    • When: The OpenAi*Model is instantiated with apiKey("demo") but without baseUrl.
    • Then: A RuntimeException should be thrown, guiding the user to specify the base URL explicitly.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: The application experiences a sudden spike in user activity.
    • Then: The HttpClient should handle the load gracefully, without exhausting resources or causing timeouts.
    • Given: The OpenAI API returns a malformed SSE response.
    • When: There is a network issue or a bug in the OpenAI API.
    • Then: The DefaultServerSentEventParser should handle the error gracefully and invoke the onError method of the ServerSentEventListener.
    • Given: A custom HttpClient implementation is provided.
    • When: The OpenAi*Model is instantiated with a custom HttpClientBuilder.
    • Then: The application should use the custom HttpClient instead of the default one.

Documentation

Changes in this PR are sufficiently documented: ✅

WIP
Copy link
Contributor

github-actions bot commented Feb 13, 2025

Click here to view this outdated comment. Please see my most recent comment below. Hi @dliubarskyi, thank you very much for your PR! ❤️ I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp. The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The removal of dev.ai4j.openai4j.OpenAiHttpException and its replacement with dev.langchain4j.exception.HttpException requires consumers to update their exception handling logic.
  • The change in default connect timeout from 60 seconds to 15 seconds might cause issues for consumers with slow network connections.
  • The removal of default modelName for OpenAiEmbeddingModel, OpenAiLanguageModel, and OpenAiStreamingLanguageModel requires consumers to explicitly set the modelName which might break existing integrations.
  • The removal of withPersisting and persistTo properties from OpenAiImageModel removes the functionality to persist generated images locally, which might break existing integrations.
  • The removal of default temperature for OpenAiLanguageModel and OpenAiStreamingLanguageModel requires consumers to explicitly set the temperature which might break existing integrations.
  • The removal of proxy configuration properties (langchain4j.open-ai.*-model.proxy.*) requires consumers to configure the proxy directly on the HTTP client, which might break existing integrations.
  • ⚠️ All the OpenAi*Model constructors are now accepting a builder object instead of all the properties
  • The apiKey is now optional for all OpenAi*Models

Potential Design Issues

  • The increased complexity of configuring the HTTP client might make the API harder to use for some consumers.
  • The removal of the withPersisting and persistTo properties from OpenAiImageModel might limit the flexibility of the API for consumers who need to persist generated images locally.
  • The introduction of builder pattern for all OpenAi*Model constructors might increase the verbosity of the code.

Potential Bugs

  • The TODO item to run all Disabled tests indicates that the changes might not be fully tested, potentially leading to bugs.
  • The migration to HttpClient might introduce new bugs related to HTTP request handling, especially around edge cases like connection pooling, SSL configuration, and request/response encoding.
  • The removal of default values for certain parameters might lead to unexpected behavior if consumers rely on those defaults.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and prompt are provided.
    • When: The chat() method is called on the OpenAiChatModel.
    • Then: The OpenAI API should return a successful response with the generated text.
    • Given: A valid API key and a list of chat messages are provided.
    • When: The chat() method is called on the OpenAiChatModel with a list of ChatMessages.
    • Then: The OpenAI API should return a successful response with the generated chat message.
    • Given: A valid API key and a tool specification are provided.
    • When: The chat() method is called on the OpenAiChatModel with a ToolSpecification.
    • Then: The OpenAI API should return a successful response with the generated text and tool execution request.

Suggested Negative Test Scenarios

    • Given: An invalid API key is used.
    • When: The apiKey is incorrect.
    • Then: An HttpException should be thrown with an appropriate error message.
    • Given: The OpenAI API is unavailable.
    • When: There is a network outage or the OpenAI API is down.
    • Then: An HttpException should be thrown after the configured number of retries.
    • Given: The user provides invalid JSON schema for tool parameters.
    • When: The JSON schema is malformed.
    • Then: An exception should be thrown during the tool execution request.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: The application experiences a sudden spike in user activity.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: The OpenAI API returns a malformed SSE response.
    • When: There is a network issue or a bug in the OpenAI API.
    • Then: The DefaultServerSentEventParser should handle the error gracefully and invoke the onError method of the ServerSentEventListener.
    • Given: A custom HttpClient implementation is provided.
    • When: The httpClientBuilder is set on the model builder.
    • Then: The application should use the custom HttpClient instead of the default one.

Documentation

Changes in this PR are sufficiently documented: ✅

@dliubarskyi dliubarskyi marked this pull request as ready for review February 13, 2025 09:41
@dliubarskyi dliubarskyi changed the title [DRAFT] OpenAI: migrate to HttpClient OpenAI: migrate to HttpClient Feb 13, 2025
WIP
Copy link
Contributor

Hi @dliubarskyi, thank you very much for your PR! ❤️
I'm a bot powered by Google AI Gemini gemini-2.0-flash-exp.
The maintainers of LangChain4j will perform a thorough code review as soon as they can, but in the meantime, here’s a preliminary review from me. I hope you find it helpful.

Potential Issues

Potential Breaking Changes

  • The removal of default model names and temperatures for some models requires users to explicitly set these parameters, potentially breaking existing code that relied on the defaults.
  • The removal of withPersisting and persistTo properties from OpenAiImageModel means that persisting images is no longer supported, breaking existing code that used this functionality.
  • The change from OkHttp/Retrofit to HttpClient and RestClient might expose subtle differences in behavior, potentially breaking existing code that relied on the specific behavior of the previous HTTP clients.
  • The removal of langchain4j.open-ai.*-model.proxy.* Spring Boot properties requires users to configure the proxy directly on the HTTP client, breaking existing configurations that used these properties.
  • ⚠️ All the OpenAi*Model constructors are now accepting a builder object instead of all the properties

Potential Design Issues

  • The increased flexibility in configuring the HTTP client might also increase the complexity of the API, making it harder for new users to get started.
  • The removal of default values might make the API more verbose and less convenient to use in simple cases.

Potential Bugs

  • The migration to HttpClient might introduce new bugs related to HTTP request handling, especially around error handling, timeouts, and connection management.
  • The removal of default values for model names and temperatures might lead to unexpected behavior if users forget to set these parameters explicitly.
  • The new proxy configuration might not be as straightforward as the previous Spring Boot properties, potentially leading to configuration errors.

Testing

Changes in this PR are sufficiently tested: ✅

Suggested Positive Test Scenarios

    • Given: A valid API key and model name are provided.
    • When: The OpenAi*Model is instantiated with correct credentials.
    • Then: The OpenAi*Model should successfully connect to the OpenAI API and return a response.
    • Given: A custom HttpClient is configured with a proxy.
    • When: The OpenAi*Model is instantiated with a custom HttpClientBuilder that configures a proxy.
    • Then: The OpenAi*Model should use the proxy to connect to the OpenAI API.
    • Given: The 'demo' key is used with the correct base URL.
    • When: The OpenAi*Model is instantiated with baseUrl("http://langchain4j.dev/demo/openai/v1") and apiKey("demo").
    • Then: The OpenAi*Model should successfully connect to the demo endpoint and return a response.

Suggested Negative Test Scenarios

    • Given: An invalid API key is provided.
    • When: The OpenAi*Model is instantiated with an incorrect API key.
    • Then: An HttpException should be thrown.
    • Given: The OpenAI API returns a 500 Internal Server Error.
    • When: The OpenAI API is temporarily unavailable.
    • Then: An HttpException should be thrown.
    • Given: The connection timeout is set to a very low value.
    • When: The OpenAi*Model is instantiated with a very short timeout.
    • Then: An HttpException should be thrown.

Suggested Corner Case Test Scenarios

    • Given: A large number of concurrent requests are made to the OpenAI API.
    • When: The application experiences a sudden spike in user activity.
    • Then: The HttpClient should handle the load gracefully without exhausting resources or causing timeouts.
    • Given: The network connection is unstable or intermittent.
    • When: There are temporary network outages.
    • Then: The HttpClient should retry requests according to the configured retry policy.
    • Given: A custom HttpClient implementation is provided.
    • When: The OpenAi*Model is instantiated with a custom HttpClientBuilder.
    • Then: The application should use the custom HttpClient instead of the default one.

Documentation

Changes in this PR are sufficiently documented: ✅

@dliubarskyi dliubarskyi merged commit f1a41b2 into main Feb 13, 2025
2 of 6 checks passed
@dliubarskyi dliubarskyi deleted the migrate-openai-to-http-client branch February 13, 2025 13:25
dliubarskyi added a commit to langchain4j/langchain4j-spring that referenced this pull request Feb 13, 2025
## Issue
Closes langchain4j/langchain4j#2468

## Change

This PR is a part of
langchain4j/langchain4j#2529


## General checklist
- [ ] There are no breaking changes
- [x] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [x] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [x] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [x] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
srnagar pushed a commit to srnagar/langchain4j that referenced this pull request Feb 24, 2025
## Issue
Closes langchain4j#2468


## Changes
1. This PR is a continuation of efforts started in
langchain4j#2413
2. When using `langchain4j-open-ai`, all `OpenAi*Model`s are now using
`java.net.http.HttpClient` instead of OkHttp/Retrofit by default. You
also have an option to [customize it or plug in any other HTTP
client](https://docs.langchain4j.dev/tutorials/customizable-http-client).
3. When using `langchain4j-open-ai-spring-boot-starter`, all
`OpenAi*Model`s are now using Spring's `RestClient` instead of
OkHttp/Retrofit by default. You also have an option to [customize it or
plug in any other HTTP
client](https://docs.langchain4j.dev/tutorials/customizable-http-client).
4. `apiKey` is now optional for all `OpenAi*Model`s
5. `OpenAi*Model`s can now throw
`dev.langchain4j.exception.HttpException` instead of
`dev.ai4j.openai4j.OpenAiHttpException`
6. The default connect timeout is now 15 seconds instead of 60 for all
`OpenAi*Model`s. The default read timeout stays the same (60 seconds).
7. To continue using the "demo" key, you now need to specify the
`baseUrl` explicitly:
```java
OpenAiChatModel model = OpenAiChatModel.builder()
        .baseUrl("http://langchain4j.dev/demo/openai/v1")
        .apiKey("demo")
        .build()
```
8. If you are using `Proxy`, now you need to configure it directly on
the HTTP client of your choice. For example, when using JDK's
`HttpClient`:
```java
HttpClient.Builder httpClientBuilder = HttpClient.newBuilder()
        .proxy(ProxySelector.of(new InetSocketAddress("XXX.XXX.XXX.XXX", 1234)));

OpenAiChatModel model = OpenAiChatModel.builder()
        .httpClientBuilder(JdkHttpClient.builder().httpClientBuilder(httpClientBuilder))
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o-mini")
        .build();
```
This also means that all `langchain4j.open-ai.*-model.proxy.*` Spring
Boot properties are not working any more.
9. `OpenAiEmbeddingModel`: removed default `modelName`
(`TEXT_EMBEDDING_ADA_002`), please set it explicitly now
10. `OpenAiImageModel`: removed `withPersisting` and `persistTo`
properties. Persisting images will not be supported any more.
11. `OpenAiLanguageModel` and `OpenAiStreamingLanguageModel`: removed
default `modelName` (`GPT_3_5_TURBO_INSTRUCT`) and `temperature`
(`0.7`), please set it explicitly now
12. `OpenAiModerationModel`: removed default `modelName`
(`TEXT_MODERATION_LATEST`), please set it explicitly now
13. All the `OpenAi*Model` constructors are now accepting a builder
object instead of all the properties
14. The `langchain4j-local-ai` module is now also using
`java.net.http.HttpClient` instead of OkHttp/Retrofit as it depends on
the `langchain4j-open-ai` module


## General checklist
- [ ] There are no breaking changes
- [X] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [X] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [x] I have added an example in the examples repo:
langchain4j/langchain4j-examples#143
- [x] I have added/updated Spring Boot starter:
langchain4j/langchain4j-spring#113
aseovic pushed a commit to coherence-community/langchain4j that referenced this pull request Feb 27, 2025
## Issue
Closes langchain4j#2468


## Changes
1. This PR is a continuation of efforts started in
langchain4j#2413
2. When using `langchain4j-open-ai`, all `OpenAi*Model`s are now using
`java.net.http.HttpClient` instead of OkHttp/Retrofit by default. You
also have an option to [customize it or plug in any other HTTP
client](https://docs.langchain4j.dev/tutorials/customizable-http-client).
3. When using `langchain4j-open-ai-spring-boot-starter`, all
`OpenAi*Model`s are now using Spring's `RestClient` instead of
OkHttp/Retrofit by default. You also have an option to [customize it or
plug in any other HTTP
client](https://docs.langchain4j.dev/tutorials/customizable-http-client).
4. `apiKey` is now optional for all `OpenAi*Model`s
5. `OpenAi*Model`s can now throw
`dev.langchain4j.exception.HttpException` instead of
`dev.ai4j.openai4j.OpenAiHttpException`
6. The default connect timeout is now 15 seconds instead of 60 for all
`OpenAi*Model`s. The default read timeout stays the same (60 seconds).
7. To continue using the "demo" key, you now need to specify the
`baseUrl` explicitly:
```java
OpenAiChatModel model = OpenAiChatModel.builder()
        .baseUrl("http://langchain4j.dev/demo/openai/v1")
        .apiKey("demo")
        .build()
```
8. If you are using `Proxy`, now you need to configure it directly on
the HTTP client of your choice. For example, when using JDK's
`HttpClient`:
```java
HttpClient.Builder httpClientBuilder = HttpClient.newBuilder()
        .proxy(ProxySelector.of(new InetSocketAddress("XXX.XXX.XXX.XXX", 1234)));

OpenAiChatModel model = OpenAiChatModel.builder()
        .httpClientBuilder(JdkHttpClient.builder().httpClientBuilder(httpClientBuilder))
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o-mini")
        .build();
```
This also means that all `langchain4j.open-ai.*-model.proxy.*` Spring
Boot properties are not working any more.
9. `OpenAiEmbeddingModel`: removed default `modelName`
(`TEXT_EMBEDDING_ADA_002`), please set it explicitly now
10. `OpenAiImageModel`: removed `withPersisting` and `persistTo`
properties. Persisting images will not be supported any more.
11. `OpenAiLanguageModel` and `OpenAiStreamingLanguageModel`: removed
default `modelName` (`GPT_3_5_TURBO_INSTRUCT`) and `temperature`
(`0.7`), please set it explicitly now
12. `OpenAiModerationModel`: removed default `modelName`
(`TEXT_MODERATION_LATEST`), please set it explicitly now
13. All the `OpenAi*Model` constructors are now accepting a builder
object instead of all the properties
14. The `langchain4j-local-ai` module is now also using
`java.net.http.HttpClient` instead of OkHttp/Retrofit as it depends on
the `langchain4j-open-ai` module


## General checklist
- [ ] There are no breaking changes
- [X] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [X] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [X] I have manually run all the unit and integration tests in the
[core](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-core)
and
[main](https://github.com/langchain4j/langchain4j/tree/main/langchain4j)
modules, and they are all green
- [X] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [x] I have added an example in the examples repo:
langchain4j/langchain4j-examples#143
- [x] I have added/updated Spring Boot starter:
langchain4j/langchain4j-spring#113
dliubarskyi added a commit to langchain4j/langchain4j-examples that referenced this pull request Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OpenAI: migrate to HttpClient
1 participant