Text-Generation-Inference v1.0+ new license: HFOIL 1.0

[OlivierDehaene]

Text-Generation-Inference, aka TGI, is a project we started earlier this year to power optimized inference of Large Language Models, as an internal tool to power LLM inference on the Hugging Face Inference API and later Hugging Chat. Since then it has become a crucial component of our commercial products (like Inference Endpoints) and that of our commercial partners, like Amazon SageMaker, Azure Machine Learning and IBM watsonx. At the same time, the project quickly grew in popularity and was adopted by other open source projects like Open-Assistant and nat.dev.

TGI v1.0 new license: HFOIL 1.0

We are releasing TGI v1.0 under a new license: HFOIL 1.0.
All prior versions of TGI remain licensed under Apache 2.0, the last Apache 2.0 version being version 0.9.4.

HFOIL stands for Hugging Face Optimized Inference License, and it has been specifically designed for our optimized inference solutions. While the source code remains accessible, HFOIL is not a true open source license because we added a restriction: to sell a hosted or managed service built on top of TGI, we now require a separate agreement.
You can consult the new license here.

What does this mean for you?

This change in source code licensing has no impact on the overwhelming majority of our user community who use TGI for free. Additionally, both our Inference Endpoint customers and those of our commercial partners will also remain unaffected.

However, it will restrict non-partnered cloud service providers from offering TGI v1.0+ as a service without requesting a license.

To elaborate further:

• If you are an existing user of TGI prior to v1.0, your current version is still Apache 2.0 and you can use it commercially without restrictions.

• If you are using TGI for personal use or research purposes, the HFOIL 1.0 restrictions do not apply to you.

• If you are using TGI for commercial purposes as part of an internal company project (that will not be sold to third parties as a hosted or managed service), the HFOIL 1.0 restrictions do not apply to you.

• If you integrate TGI into a hosted or managed service that you sell to customers, then consider requesting a license to upgrade to v1.0 and later versions - you can email us at api-enterprise@huggingface.co with information about your service.

Why the new license?

TGI started as a project to power our internal products, and we see it as a critical component of our commercial solutions. TGI is not meant as a community-driven project, but as a production solution that’s widely accessible to the community. We want to continue building TGI in the open, and will continue to welcome contributions. But unlike community-driven projects like Transformers and Diffusers focused on making machine learning accessible, TGI is focused on performance and robustness in production contexts, with the goal of building commercial products.

What about Hugging Face contributions to open source?

Our mission as a company is to democratize good machine learning. An important component of democratization is making good machine learning more accessible. We achieve this through community-driven open source projects like Transformers, Diffusers, Datasets, our free courses (Transformers, Diffusers Audio, RL), and many more libraries collectively garnering about 240k GitHub stars as of this writing. Our long term commitment to open source has not changed.

[dongs0104]

Hello is it similar license as Elastic License v2 at ElasticSearch?

[julien-c]

@dongs0104 correct

[Atry]

TGI depends vLLM, which is licensed under Apache. What if vLLM changes their license for the same reason and forbid TGI from using vLLM?

[OlivierDehaene]

TGI depends on our fork of VLLM which is Apache 2.0.

[shibanovp]

Transitioning to HFOIL 1.0 is understandable, but please consider introducing both free/growth and enterprise plans. It would be great if I could pass an access token to args, and have it function as a license key. I believe you could collect this and this for telemetry.

Here are the benefits I foresee:

1. Your sales team wouldn't have to spend time discussing "We are innovative ... with 0.5 active monthly users", and could instead focus on taliking with qualified prospects.
2. Startups with pre-configured billing could effortlessly use TGI.

[OlivierDehaene]

@shibanovp I think you are misunderstanding the terms of the license.

You can continue using TGI for free without ever interacting with our sales team if the service your are building or selling is not a container serving engine tighly integrated with TGI (for example a serverless TGI experience where user can pick a hub model and it automatically starts a TGI container for them like we offer in Inference Endpoint).

Then AND ONLY THEN would you need to contact us to get a license if AND ONLY IF you want to use TGI 1.0+.

To summarize:

• building and selling a chat app for example that uses TGI as a backend is ok whatever the version you use
• building and selling a Inference Endpoint like experience using TGI 1.0+ requires an agreement with HF

[rom1504]

I think if you write in the readme that is not opensource software then contributors will not be confused.

[OlivierDehaene]

@rom1504 all contributors that we could contact received this information days in advance.

We warn new contributors about the license change directly in their PR when reviewing them (like in #617) and we might add a proper Contributor License Agreement in the future.

Some contributors decided to close their PR given the new license (#529 #719).

What type of language would you like to see in the README?

[shibanovp]

@OlivierDehaene
Thank you for the clarification. I do have a question, however. Can the self-service I'm selling allow clients to select a fine-tuned model (based on their private data), and create / destroy it with Terraform on Kubernetes in either a public cloud or on-premise that includes TGI as part of the stack?

[OlivierDehaene]

@shibanovp I do not think I am the best interlocutor to answer such a precise question.

Are you ok with contacting us at api-enterprise@huggingface.co to loop in our legal team?

[shibanovp]

Are you ok with contacting us at api-enterprise@huggingface.co to loop in our legal team?

I thought as much...

This is what I was referring to in #726 (comment)

The workflow I would prefer is as follows:

1. I add a credit card.
2. I pass an access token to args.
3. all works

However, the current workflow is:

1. I have to prepare for the v1.0 migration.
2. I must contact your team.
3. I follow X number of steps.
4. all works

[rom1504]

@OlivierDehaene "This tool is not opensource. We provide open access via license XYZ"