MFG602716

DIY: Create a ChatGPT-Based App to Control Inventor

with Natural Language
Kivanc Kizildemir
Codeo Solutions Limited

Learning Objectives
• Learn about OpenAI’s GPT-4 model, particularly ChatGPT.
• Learn to integrate ChatGPT into an application for Inventor.
• Discover the process of creating a ChatGPT-based application, including
troubleshooting.
• Discover and learn how to overcome potential challenges in developing a
ChatGPT-based Inventor application.

Description
In the session, we will explore the fusion of artificial intelligence (AI) and CAD, demonstrating
how you can use the power of OpenAI's language model, ChatGPT, to interact with Inventor
software. With the increase in AI capabilities, the realm of CAD software has been
revolutionized, and this session aims to showcase how natural language can be used to
manipulate and control design elements within Inventor. Starting with an overview of the OpenAI
GPT-4 model and its applications, we will delve into how ChatGPT can be integrated into an
application to control Inventor using natural language. We’ll walk through the process of creating
an application, highlighting key areas like API integration, user interaction, and troubleshooting,
and we’ll discuss potential challenges and solutions. You’ll also get hands-on experience with
the model during the session, allowing you to better understand the application process.

Speaker
As a Mechanical Engineer and CAD&CAE Specialist, I hold certifications
as an Autodesk Expert Elite, Certified Instructor, and Inventor Certified
Professional. I've had the privilege of contributing to a diverse array of
projects, from military vehicles to weighing automation systems. Currently,
I serve as Director at Codeo Solutions, where I oversee 3D configurator
development and offer 3D-based digital transformation services globally.
I'm a frequent speaker at Autodesk University and am committed to
advancing the field through knowledge sharing.

Feel free to contact me for any questions about this class:

https://www.linkedin.com/in/kivanckizildemir/
[email protected]

Page 1
Overview of OpenAI's GPT-4
GPT-4, the Generative Pre-trained Transformer model, is the newest creation from OpenAI's
lineup of artificial intelligence tools. With an impressive 1.7 trillion machine learning parameters,
GPT-4 isn't just a simple text generation tool. It represents a culmination of advanced research
and technology, being proficient in a plethora of tasks, extending way beyond natural language
comprehension and generation.

Working Mechanism
The strength of GPT-4 lies in its architecture. Using transformers, GPT-4 processes and
understands the context from sequences of text tokens. Unlike traditional models that might
work linearly, transformers allow for parallel processing of token sequences, capturing
contextual information more effectively. This design enables GPT-4 to make highly accurate
predictions for the next word in a sequence. On the surface, this capability might seem basic,
but when scaled, it becomes a potent tool, capable of creating coherent and contextually
relevant paragraphs of text.

Applications of GPT-4
GPT-4's versatility is truly remarkable:.

Code Writing
GPT-4 can assist in writing code snippets, making the lives of developers easier.

Conversational Agents
Think of chatbots or virtual assistants; GPT-4 can drive more human-like interactions.

Complex Design Assistance

In the realm of CAD and other design tools, GPT-4 can aid in automating or suggesting
design elements.

Content Creation
Be it blog posts, articles, or creative writing, GPT-4 can generate content that is both
coherent and contextually appropriate.

Language Translation
With its deep understanding of multiple languages, GPT-4 can be used for real-time
translation services.

... and many more!

Understanding Parameters
The power of GPT-4 isn't just in its massive parameter count but also in the user's ability to
guide its behavior. Parameters like temperature and max tokens allow users to influence the
randomness and length of the generated text, respectively. This means that while GPT-4
operates on learned data, users still retain control, ensuring that the generated content aligns
with their specific needs.

Page 2
Key Parameters
Temperature:
Effect
Influences randomness of the output.

Examples
A value of 0.9 leads to more random output.
A value of 0.2 results in more focused, deterministic output.

Use Cases
Ensuring technical accuracy or fostering creativity.

Max Tokens:
Effect
Limits output length.
Examples
Setting it to 50 limits the output to 50 tokens.

Top P:
Effect
Controls output diversity by token selection.
Explanation
Limits token selection to those with a cumulative probability of at least top_p.
Use Cases
Controlling the variety of tokens in the output.

Frequency Penalty:
Effect
Penalizes tokens based on training data frequency.
Explanation
Manages the likelihood of high-frequency or low-frequency words in the output.

Presence Penalty:

Effect
Determines if new tokens should appear.
Examples
Positive value: Encourages using new tokens.
Negative value: Discourages using new tokens.

Page 3
Walkthrough: Creating the App
We've covered the theory, now let's dig into the practical steps for creating a third-
party form application to control Inventor through ChatGPT. Please see the main
workflow that we have to go through:

Development Environment Setup:

The first step is to design the Form App UI, which can be done using WinForms in Visual
Studio. You'll also need to install certain software and libraries, including Inventor itself and
possibly a package for HTTP requests in C#. Then we initialize Inventor Interop, which allows
our C# application to interact with Inventor software.
API Interaction with ChatGPT:
To interact with ChatGPT, you can use C#'s HttpClient class to make API calls. Once
you get the JSON response, you can parse it using the Newtonsoft.Json library to
extract the text generated by ChatGPT.
Text-to-Command Conversion:
The next step is crucial: converting the received text into something that can
be executed as a command in Inventor. This involves post-processing the
text and compiling it into actual commands or function calls that the Inventor
software can understand. We’ll be using CSharpCodeProvider as compiler.
Execution and Validation:
Now, it's time to run these commands using Inventor Interop. This is where the rubber meets the
road. After execution, it’s important to validate that the action in Inventor matches what was
intended. This might involve programmatically checking the model's state or dimensions.

Page 4
Testing and Debugging:
Last but not least, you should run comprehensive tests to see how well your application
performs. Implement robust error-handling to manage issues that might arise during the
interaction between ChatGPT and Inventor, or from user input errors.

That concludes our walkthrough. By following the following steps, you will be well-equipped to
build a functioning third-party form application to control Inventor via natural language, powered
by ChatGPT.

Step-by-Step Guide
1. Setup Environment:

• Install Microsoft Visual Studio and Autodesk Inventor.

2. Create a New Project:

• Launch Visual Studio, create a new ‘Windows Forms App (.NET Framework)’ project
named ‘ChatGPTAU2023’

namespace ChatGPTAU2023
{
public partial class MyForm : Form
{
// ...
}
}

3. Install Required Packages:

• Navigate to ‘Project’ → ‘Manage NuGet Packages’.

• Click on ‘Browse’, search for ‘Newtonsoft.Json’, select it, and click ‘Install’.
• Search for ‘Microsoft.CodeDom.Providers.DotNetCompilerPlatform’, select it, and
click ‘Install’.

4. Design the User Interface:

• In the ‘Form Designer’, add two TextBoxes (‘inputTextBox’ and ‘outputTextBox’)
and a Button (‘submitButton’). Arrange them neatly.

5. Rename Form1 to MyForm:

• In the Solution Explorer, rename ‘Form1.cs’ to ‘MyForm.cs’.

• Update the class name within the code to ‘MyForm’.

Page 5
public partial class MyForm : Form
{
// ...
}

6. Obtain an OpenAI API Key:

• Sign up on OpenAI’s website.

• Follow the instructions to create a new API key.

7. Define API Key Constant:

string API_KEY = "your-api-key";

8. Create Constructor Method:

public MyForm()
{
InitializeComponent();
}

9. Create submitButton_Click Method:

private async void submitButton_Click(object sender, EventArgs e)

{
try
{
string request = inputTextBox.Text;
if (request.Contains("\n"))
request = request.Replace("\n", " ");
// ...
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}

10. Preparing the HTTP Request:

string request = inputTextBox.Text;

HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("authorization", "Bearer " + API_KEY);

11. Sending Request to GPT-4:

var response2 = await client.PostAsync(endpoint, content);

Page 6
string responseContent = await response2.Content.ReadAsStringAsync();

12. Parsing and Displaying the Response:

var jsonResponse = JObject.Parse(responseContent);

var response = jsonResponse["choices"][0]["message"]["content"].Value<string>();
outputTextBox.Text = response;

13. Adjusting and Saving the Generated Code:

string input = response;

if (input.IndexOf("```csharp") != -1)
input = input.Substring(input.IndexOf("```csharp") + 10);
else if (input.IndexOf("```") != 1)
input = input.Substring(input.IndexOf("```") + 4);

if (input.IndexOf("```") != -1)
input = input.Substring(0, input.IndexOf("```"));

if (!input.Contains("using Inventor;\n"))
input = "using Inventor;\n" + input;

if (!input.Contains("using System;\n"))
input = "using System;\n" + input;

if (input.Contains("Document newDoc = invApp.Documents.Add(DocumentTypeEnum.kPartDocumentObject,

false);"))
input = input.Replace("Document newDoc =
invApp.Documents.Add(DocumentTypeEnum.kPartDocumentObject, false);", "Document newDoc =
invApp.Documents.Add(DocumentTypeEnum.kPartDocumentObject);");

StreamWriter sw = new StreamWriter("Input.txt");

sw.WriteLine(input);
sw.Close();

14. Setting up CSharpCodeProvider:

CSharpCodeProvider provider = new CSharpCodeProvider();

CompilerParameters parameters = new CompilerParameters
{
GenerateExecutable = false,
GenerateInMemory = true
};

15. Compiling and Executing the Generated Code:

CompilerResults results = provider.CompileAssemblyFromSource(parameters, input);

// ... code to execute or handle compilation errors ...

Page 7
 16. Handling Compilation Errors:

if (results.Errors.HasErrors)
{
foreach (CompilerError error in results.Errors)
{
outputTextBox.Text += "\n" + error.ToString();
}
}

17. Executing the Generated Code:

else
{
Assembly assembly = results.CompiledAssembly;
Type[] types = assembly.GetTypes();
MethodInfo method = types[0].GetMethod("Main");
method.Invoke(null, null);
}

18. Adding Referenced Assemblies:

• This new step will ensure that the required assemblies are referenced when compiling
the generated code.

parameters.ReferencedAssemblies.Add(@"C:\Program Files\Autodesk\Inventor 2021\Bin\Public

Assemblies\Autodesk.Inventor.Interop.dll");
parameters.ReferencedAssemblies.Add(@"C:\Program Files (x86)\Reference
Assemblies\Microsoft\Framework\.NETFramework\v4.7.2\System.dll");
parameters.ReferencedAssemblies.Add(@"C:\Program Files (x86)\Reference
Assemblies\Microsoft\Framework\.NETFramework\v4.7.2\System.Windows.Forms.dll");

19. Adjusting Main Method Visibility:

• This new step includes the snippet to adjust the visibility of the Main method in the
generated code.

if (!input.Contains("public static void Main()") && input.Contains("static void Main()"))

input = input.Replace("static void Main()", "public static void Main()");
else if (!input.Contains("public static void Main(string[] args)") && input.Contains("static void
Main(string[] args)"))
input = input.Replace("static void Main(string[] args)", "public static void Main()");

Page 8
Testing Your Application
1. Run Autodesk Inventor
2. Run Your Application:
• Press F5 to run your application in Visual Studio.
3. Enter a Request:
• In ‘inputTextBox’, enter a request like "create a new part."
4. Generate Code:
• Click ‘submitButton’ to generate code. Check ‘outputTextBox’ for generated code.
5. Review & Execute Code:
• Review the generated code. If it looks correct, the code will execute automatically,
creating a new part in Inventor.

Future Works
The frontier of integrating AI with CAD software is vast and continually evolving. The successful
fusion of ChatGPT with Autodesk Inventor opens numerous avenues for further exploration and
development. Here are some potential directions for future work in this exciting intersection:

Training ChatGPT
Training ChatGPT on domain-specific datasets can significantly enhance its understanding and
generation capabilities regarding CAD operations and terminologies. A more specialized model
could provide more accurate and sophisticated interactions with Inventor, thereby enabling the
creation of more complex designs through natural language commands.

Uploads and Image Processing

Expanding the application to handle file uploads could allow users to input existing designs,
images, or sketches, which ChatGPT could help interpret, modify, or enhance. Furthermore,
integrating image recognition and processing techniques could enable the application to convert
sketches or images into corresponding Inventor designs, offering a more intuitive way for users
to express their design ideas.

Working with Datasets

Incorporating datasets within the application could allow for data-driven design and analysis. For
instance, users could input datasets to drive the generation of parametric designs, or to perform
simulations and optimizations based on real-world data. This would significantly broaden the
scope and capabilities of the application, making it a powerful tool for data-driven design and
engineering.

Extending to Other CAD Platforms

The principles and techniques learned from integrating ChatGPT with Inventor could potentially
be extended to other CAD platforms. This cross-platform capability could lead to the
development of a unified AI-driven interface for multiple CAD systems, greatly simplifying the
workflow for designers and engineers working with multiple software tools.

Page 9
These future work directions offer a glimpse into the vast potential and the myriad of possibilities
that lie ahead in the fusion of AI and CAD. As we continue to explore and innovate, the horizon
of what can be achieved expands, heralding a new era of design and engineering powered by
AI.

Code Repository:

For further exploration and a hands-on experience, attendees are encouraged to visit the
GitHub repository hosting the project's source code and additional resources.

GitHub Repository: https://github.com/kivanckizildemir/InventorChatGPT

Here, you will find the complete source code used in this session, along with additional
documentation and examples to help you get a deeper understanding of building a ChatGPT-
based app to control Inventor with natural language.

Page 10