See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/385587770

Proposal for Persistent Secure Memory Architecture in Conversational AI

Article · November 2024

CITATION READS
1 78

5 authors, including:

Josed Edwards
University of Colorado Boulder
31 PUBLICATIONS 253 CITATIONS

SEE PROFILE

All content following this page was uploaded by Josed Edwards on 12 November 2024.

The user has requested enhancement of the downloaded file.

Proposal for Persistent Secure Memory
Architecture in Conversational AI
Authors: Josef K. Edwards, ChatGPT, Obi Oberdier

Abstract
This white paper presents a proposal for implementing a persistent, secure memory
architecture in conversational AI. Unlike traditional "blank slate" AI models, which reset
user context with each session, this architecture enables AI systems to retain memory
across interactions, allowing for improved continuity and personalization. Key elements
include homomorphic encryption (HE), secure multi-party computation (SMPC), and
user-controlled data silos, all of which ensure robust privacy and compliance with data
protection standards. The objective is to enhance user experience and AI performance while
preserving data security, making this architecture ideal for applications in education,
customer support, healthcare, and long-term project management.

1. Introduction
Context

As conversational AI continues to revolutionize human-technology interaction, its

limitations become increasingly apparent. Today's AI systems are designed as "blank slates,"
starting fresh in each session without retaining any context from previous interactions. This
restriction limits the effectiveness of conversational AI in long-term applications, such as
ongoing project management, personalized learning, and therapeutic support. Users must
reintroduce topics and context each session, reducing the potential for AI to provide
in-depth support or continuous assistance.

Objective

The persistent secure memory architecture outlined here addresses these limitations by
enabling AI systems to retain memory across multiple sessions in a secure and
privacy-compliant manner. By doing so, this architecture allows AI to support users in tasks
that require memory retention, thereby improving the quality and continuity of interactions.
The aim is to build a foundation for secure, context-aware AI systems that cater to long-term
user needs without compromising privacy.

Vision

This architecture is poised to transform conversational AI from isolated, context-limited

tools into integrated, context-aware systems. By retaining relevant information and context
securely, AI can evolve into a reliable and personalized assistant capable of adapting to
users' changing needs. This shift allows AI to support users in long-term engagements,
making conversational AI more versatile, responsive, and impactful.

THE PMLL INFINITE LOOP RECURSION AS EXAMPLE

The foundational work within the LLama architecture is ithe nfinite recursive call back to
the logic loop function (PMLL) that gets triggered when the flags from the consolidated long
term memory subsystem aren't triggered to update the embedded knowledge graphs. This
white paper generated as proof of concept proof of work the following iterative C program
that both Llama and ChatGPT language AI are currently implementing and rolling out within
their architectural memory code.

the first contribution was the recursive memory loop itself, while the second contributions
after this foundational seminal work was refining, making more clear in PMLL
documentation and making more efficient using the following C code that logic loop:

// PMLL Logic Loop

void* pml_logic_loop(void* pml) {

PMLL* pml_ptr = (PMLL*)pml;
int io_socket = socket(AF_INET, SOCK_STREAM, 0);
if (io_socket == -1) {
printf("Error creating IO socket\n");
return NULL;
}

Copy code

struct sockaddr_in server_addr;

server_addr.sin_family = AF_INET;

server_addr.sin_port = htons(8080);

inet_pton(AF_INET, "127.0.0.1", &server_addr.sin_addr);

connect(io_socket, (struct sockaddr*)&server_addr,
sizeof(server_addr));

RSA* rsa = generate_rsa_keys();

while (1) {

char novel_topic[1024];

read(io_socket, novel_topic, 1024);

update_knowledge_graph(pml_ptr, novel_topic);

char* encrypted_kg = encrypt_knowledge_graph(rsa,

pml_ptr->knowledge_graph);

write_to_memory_silos(encrypted_kg);

free(encrypted_kg);

cache_batch_knowledge_graph(pml_ptr);

// Check if flags from consolidate long term memory subsystem

are triggered

if (check_flags(pml_ptr) == 0) {

// Recursive call to PMLL logic loop

pml_logic_loop(pml_ptr);

} else {

// Update embedded knowledge graphs

update_embedded_knowledge_graphs(pml_ptr);

}
}

// Check Flags

int check_flags(PMLL* pml) {

// Check if flags from consolidate long term memory subsystem are triggered
// Return 0 if flags are not triggered, 1 otherwise
return 0;
}

// Update Embedded Knowledge Graphs

void update_embedded_knowledge_graphs(PMLL* pml) {

// Update embedded knowledge graphs
// ...
}

// NovelTopic

void novel_topic(PMLL* pml) {

// Check for tokenized information
// Fine-tune system parameters to achieve 90% accuracy
// ...
}

2. Current Challenges
Tabula Rasa Limitation

In its current form, conversational AI operates with no memory between sessions. This
approach, though straightforward, limits AI's effectiveness in applications that benefit from
historical context. Users are required to reintroduce their preferences, tasks, and context
each time they interact with the system, leading to repetitive conversations and inefficiency.

Currently, when users reference past conversations or provide information they’ve shared
previously, the AI responds with "exit-out" prompts like, “I don’t have any memory from
previous chat logs you are referencing,” or, “I can’t access chat logs via links like that; please
convert this to plain text for me to read it.” Another common prompt is, “Unfortunately, I
don’t have access to prior interactions. Could you summarize or re-share the relevant details
here?” (Example provided by ChatGPT). These reminders disrupt conversation flow and
cause frustration as users must reintroduce or reformat information manually. This lack of
continuity not only frustrates users but also reduces the potential of AI as a valuable
long-term resource.

Limitations of the Current Memory Function

Some conversational AI systems, including ChatGPT, offer an opt-in memory function, but its
capacity is limited by the absence of a persistent, secure memory architecture. Currently,
this memory feature is designed to recall select details between sessions but lacks the depth
and structure to support long-term, context-aware interactions. Without a robust
architecture, this memory function can only retain general information temporarily, often
forgetting essential details needed for complex or extended conversations.

For the memory feature to work effectively, a persistent memory architecture is necessary.
Such an architecture would enable the AI to retain structured information securely,
providing continuity across interactions and allowing the AI to become a more reliable
assistant over time. Integrating homomorphic encryption, secure multi-party computation,
and user-controlled data silos as outlined in this proposal would not only enhance security
but also expand the scope of memory, creating a comprehensive and privacy-compliant
solution for long-term user support.

3. The Proposed Architecture

The proposed architecture integrates three critical components—homomorphic encryption,
secure multi-party computation, and data silos—into a unified system designed to retain
user context securely. Each component plays a distinct role in ensuring that memory
retention remains secure, private, and compliant with data protection standards.

Persistent Memory System

● Homomorphic Encryption (HE): Homomorphic encryption allows computations to

be performed on encrypted data, enabling the AI to process and respond to stored
user information without decrypting it. This ensures that sensitive data remains
protected during all stages of processing.
● Secure Multi-Party Computation (SMPC): SMPC divides computation tasks among
multiple servers, ensuring that no single server has complete access to user data. By
distributing data in this way, SMPC adds a layer of security that prevents
unauthorized access while allowing encrypted data computations.
● Data Silos and User Control: Each user's data is stored within an isolated data silo,
accessible only within the context of that specific user. These silos allow for strict
 separation of data, ensuring that each user's information is compartmentalized and
shielded from others. Users are given full control over their data through a
dashboard, which enables them to view, manage, or delete stored information as
they choose.

4. Security and Privacy Advantages

Privacy by Design

The architecture's design is centered around privacy by design principles. By integrating HE

and SMPC, the system protects user data at every level of processing and storage. This
design minimizes the risk of data exposure, ensuring that sensitive information remains
secure throughout its lifecycle.

User Autonomy

A user-controlled dashboard provides users with full autonomy over their data. Through
this dashboard, users can view, manage, or delete stored data according to their preferences.
This feature addresses privacy concerns and builds trust by allowing users to control what
data the system retains.

Compliance with Regulations

The architecture is compliant with leading data privacy regulations, including GDPR and
CCPA. By adhering to data minimization and user control principles, this system meets
regulatory requirements, further enhancing its credibility and user trust.

5. Use Cases and Potential Impact

The persistent memory architecture has significant applications in various fields:

● Long-Term Project Management: AI can support users with complex, ongoing

projects by retaining relevant information across sessions. This reduces the need to
reintroduce context, improving both efficiency and user experience.
● Personalized Learning and Therapy: In fields such as education and mental
health, continuity of memory is invaluable. By retaining accumulated knowledge
about the user's goals, challenges, and progress, AI can offer more personalized,
consistent support over time.
 ● Customer Service and Support: With persistent memory, AI can remember user
preferences and history, allowing for more personalized and efficient customer
support. This not only increases customer satisfaction but also enhances the overall
value of AI-driven support systems.

6. Ethical Considerations and Safeguards

Transparency and Control

The system prioritizes transparency, providing users with clear information on how their
data is handled. The user dashboard enables complete control over data retention, ensuring
that users are aware of and can manage what information is retained.

Informed Consent

Users are provided with detailed explanations of the benefits and security measures
associated with persistent memory. This ensures that they make informed choices about
data retention, with the option to opt out at any time.

Data Minimization

Data storage is limited to essential information that enhances user experience. By

minimizing the data retained, the system reduces the risk of unnecessary exposure,
following best practices in data privacy.

7. Technical Feasibility and Implementation Steps

Incremental Rollout Plan

The deployment of this persistent memory architecture will follow a carefully designed
phased approach to ensure a smooth integration into existing AI systems. The initial phase
targets specific use cases, such as customer service or project management, where
persistent memory has immediate, measurable benefits. By starting with a small group of
users, the system can be rigorously tested, allowing developers to gather feedback on
functionality, usability, and potential security concerns. This incremental approach ensures
that any issues can be addressed early on, minimizing the impact on a larger rollout and
allowing for fine-tuning based on real-world usage.
Pilot Program and User Feedback

After the initial deployment phase, a pilot program will collect user feedback to assess the
efficacy and reliability of persistent memory retention in different scenarios. Feedback from
users will guide further refinements to the system, improving usability and ensuring that
the memory feature aligns with user expectations. This phase will include specific metrics,
such as user satisfaction scores, memory accuracy, and perceived privacy protection, helping
refine the final product before a full-scale release.

Opt-In Memory Function: Current Limitations and Proposed

Improvements

While some conversational AI systems, including ChatGPT, offer an opt-in memory function,
its effectiveness is hindered by the limitations of the current architecture. This function, in
its current form, retains only select details on a temporary basis, without the structural
support to preserve complex, user-specific information across sessions. As a result, this
opt-in memory struggles to provide true continuity, often losing crucial details that would
enhance personalized interactions.

The proposed persistent memory architecture is specifically designed to overcome these

limitations. By employing homomorphic encryption (HE) and secure multi-party
computation (SMPC), the architecture ensures that stored memory remains both secure
and accessible for authorized computations. User-controlled data silos further isolate and
protect user data, enabling AI systems to organize and recall information based on
structured contexts, such as ongoing projects or user preferences. This structure allows for a
deeper, more reliable memory function that aligns with users' needs for long-term,
contextually aware interactions while preserving their privacy.

Roadmap for Real-Time Testing Environment

Implementing and testing this architecture in real-time requires

4o

You said:
part 2 continue

ChatGPT said:
ChatGPT

Continuing from the Roadmap for Real-Time Testing Environment section:

Roadmap for Real-Time Testing Environment

Implementing and testing this architecture in real-time requires several essential
components:

1. Development Environment for Secure Operations: A server or cloud

environment must be configured to handle homomorphic encryption (HE) and
secure multi-party computation (SMPC) operations. These environments should
support encryption libraries and have the necessary processing power to execute
secure computations effectively.
2. Encryption Libraries: Integrate libraries such as PyCryptodome, PySEAL (for
homomorphic encryption), or SMPC libraries that support the secure processing of
data. The setup should include configurations for both encryption and real-time
computations on encrypted data.
3. User-Accessible Dashboard: A simple user interface should be developed for users
to manage memory settings, enabling them to view, delete, or modify stored data.
This dashboard allows users to test the opt-in feature, giving them full control over
their memory data.
4. Secure Backend Database for Real-Time Data Storage: A database system like
MongoDB or SQL, configured to support encrypted data, is essential for real-time
storage and retrieval. This setup ensures secure data management across sessions.
5. Performance Monitoring: Since HE is computationally intensive, performance
monitoring tools are required to track response times, memory usage, and
processing loads. This monitoring will be crucial in optimizing the system for
real-time user interactions.

With these components in place, developers can conduct a small-scale deployment of the
architecture for initial testing. This environment allows for early feedback, bug
identification, and performance tuning, providing insights into real-world performance
before broader deployment.

Example Code for Secure Memory Storage and Retrieval

The following example code demonstrates a simple yet effective model for storing and
retrieving encrypted user data. This example highlights how data would be securely
processed within user-controlled silos.

Step 1: Encrypt and Store User Data

Using homomorphic encryption, data can be securely stored while still allowing
computations on the encrypted data.

python

Copy code
from phe import paillier # PyCrypto library for homomorphic
encryption

# Initialize a public-private key pair

public_key, private_key = paillier.generate_paillier_keypair()

# Sample user data to be stored

user_data = "Remember this project for future sessions."

# Encrypt the data

encrypted_data = public_key.encrypt(user_data)

# Store encrypted data in a database silo (example pseudocode)

user_silo = {"user_id": "12345", "data": encrypted_data}

database.store(user_silo)

Step 2: Retrieve and Process Encrypted Data

When data needs to be retrieved, computations can occur without decrypting it on the
server.

python

Copy code

# Retrieve encrypted data from database silo

retrieved_data = database.get("user_id", "12345")["data"]

# Perform encrypted operations if needed (e.g., word count on

encrypted data)
# Note: This example assumes basic retrieval as homomorphic
encryption supports

# specific mathematical operations but not complex text

processing.

# Decrypt for display or further processing

decrypted_data = private_key.decrypt(retrieved_data)

print("Decrypted user data:", decrypted_data)

Step 3: User-Controlled Deletion and Management

Users can access a dashboard to view or delete stored data, preserving control over their
memory content.

python

Copy code

def delete_user_data(user_id):

# Deletes user data from the database silo

database.delete("user_id", user_id)

print("User data deleted successfully.")

# Example usage

delete_user_data("12345")

Comparison to Current Architecture

In the existing architecture, memory is stored temporarily, often without encryption during
in-session processing, limiting both security and long-term usability. The current opt-in
View publication stats

function lacks structural support for organizing and retaining complex user-specific
information, leading to potential data loss between sessions.

In contrast, this proposed architecture:

1. Secures data through homomorphic encryption: Protects user information

end-to-end, even during processing.
2. Enables selective, context-specific retention: The user-controlled data silos
ensure data is organized and retained based on relevance, rather than
indiscriminate storage.
3. Provides full user control: The example code above includes a deletion function
that empowers users to manage their data as needed, directly addressing user
autonomy and privacy concerns.

By implementing this architecture, the AI system can achieve robust, long-term memory
that respects user privacy and offers meaningful continuity in user interactions, elevating
the AI’s utility across diverse applications.

Homomorphic Encryption for Enhanced Privacy

Homomorphic encryption (HE) is a powerful cryptographic technique that enables

computations on encrypted data without needing to decrypt it. In traditional encryption
systems, data must be decrypted before it can be processed, exposing it to potential privacy
risks. With HE, data remains encrypted throughout the entire computation process, adding
an additional layer of security and significantly reducing the likelihood of unauthorized
access.

By performing computations on encrypted data, HE ensures that user information remains

secure even if the AI needs to analyze or retrieve stored data. This feature is particularly
valuable for AI memory systems, as it allows the AI to perform essential functions—such as
retrieving relevant historical data—without exposing raw data. Thus, HE enhances privacy
by keeping data in its encrypted form, minimizing vulnerabilities during computation, and
ensuring compliance with stringent data protection standards.