Building Permission-Aware RAG for Enterprise Knowledge Systems

Building a Retrieval-Augmented Generation (RAG) application becomes much more challenging when it moves from a prototype to an enterprise environment.

In a simple RAG application, documents are processed, converted into embeddings and retrieved when a user asks a question. In an enterprise system, however, knowledge may be distributed across Google Drive, SharePoint, Slack, Jira, Confluence and many other platforms. More importantly, different employees have access to different information.

This creates an important requirement: the AI must not simply find the most relevant information. It must find the most relevant information that the current user is authorized to access.

While developing enterprise knowledge and AI systems at Web Foundation, we have worked extensively on this problem. This article describes some of the architecture and engineering lessons from building permission-aware RAG systems in practice.

Connecting Enterprise Knowledge From Multiple Systems

Organizational knowledge rarely exists in one place. Documents may be stored in cloud drives, conversations in communication tools, tasks in project-management platforms and business information in other applications.

Our integration layer has included systems such as Google Drive, Gmail, Google Calendar, Asana, Dropbox, OneDrive, SharePoint, Outlook, Slack, Jira, Confluence, Notion and Trello.

All of these integrations are read-only. The knowledge platform retrieves and processes information from the source systems without modifying or deleting the original data.

Supporting many integrations creates its own engineering challenges. Each platform has different APIs, authentication mechanisms, rate limits, data structures and synchronization models. A reliable RAG system therefore needs much more than an LLM and a vector database; it also needs a robust data synchronization layer.

Why Enterprise RAG Must Understand Permissions

Consider two employees asking exactly the same question.

One employee may have access to confidential management documents while another only has access to public team documentation. Even though their questions are identical, the information available to the AI should be different.

This means retrieval cannot be based only on semantic similarity.

For supported document sources such as Google Drive, Dropbox and OneDrive, we replicate document access permissions into the knowledge system. During retrieval, these permissions become part of the filtering process so that candidate chunks are limited to information the requesting user is authorized to access.

As a result, two users can ask the same question and receive different answers because their accessible organizational knowledge is different.

This is an important distinction between a general RAG prototype and an enterprise knowledge system.

Keeping Permissions Synchronized

Copying permissions during the initial document import is not sufficient.

A document that a user can access today may be restricted tomorrow. A new employee may be granted access to a folder, while another employee may lose access to it.

If the knowledge system does not reflect those changes quickly, its authorization state becomes different from the source system.

Where supported, we use webhooks to detect changes in connected platforms and synchronize updated document and permission information. This allows the knowledge system to respond to changes in the original source instead of treating permissions as static metadata captured during ingestion.

The goal is straightforward: access inside the AI knowledge layer should follow access in the original business systems as closely as possible.

From Documents to Searchable Knowledge

After information is retrieved from a connected source, it goes through a processing pipeline before becoming available to the RAG system.

At a simplified level, our pipeline follows these steps:

1. Retrieve information from the connected source.
2. Extract and normalize the text.
3. Split the content into chunks of approximately 1,000 tokens.
4. Create an embedding for each chunk.
5. Store the embeddings together with document metadata and access information.
6. Retrieve relevant, authorized chunks when a user asks a question.
7. Provide the retrieved context to the language model to generate the answer.

We have used Amazon Bedrock with the Titan Text Embeddings V2 model for embedding generation. The vector data is stored directly in PostgreSQL using pgvector together with pgvectorscale.

Keeping relational data, permissions, document metadata and vectors close together in PostgreSQL also gave us useful architectural flexibility when building permission-aware retrieval.

Scaling Vector Search Without Keeping the Entire Index in RAM

Vector search performance became one of the areas where we spent significant engineering and research time.

As the number of chunks grows, the design of the vector index has a major impact on both query performance and infrastructure requirements. Keeping increasingly large vector indexes in memory can make infrastructure considerably more expensive as the knowledge base grows.

We therefore explored pgvectorscale and its StreamingDiskANN index.

One of the characteristics that made this approach interesting for our workload was its ability to use disk-based storage as part of approximate nearest-neighbor search rather than requiring the complete vector index to remain in RAM.

This creates an important infrastructure trade-off for large knowledge bases: maintaining fast vector retrieval while reducing the amount of memory required by the database server.

Choosing a vector-search architecture therefore should not be based only on which database can store embeddings. Index behavior, memory consumption, filtering requirements and expected dataset size all matter when moving toward production workloads.

Combining Vector Search With Permission Filtering

Fast semantic search alone does not solve enterprise retrieval.

The system must efficiently answer two questions at the same time:

Which chunks are semantically relevant to this question?

and:

Which of those chunks is this user allowed to access?

Permission information therefore becomes part of the database retrieval strategy rather than something checked only after an unrestricted vector search.

We use PostgreSQL indexing, including GIN indexes for appropriate permission-related data, together with vector retrieval to reduce the amount of information that needs to be evaluated during a request.

This required considerable testing because aggressive filtering can reduce retrieval quality, while retrieving too much data and filtering it later can increase latency and create unnecessary processing.

The architecture needs to balance authorization, semantic relevance and performance.

Keeping AI Responses Fast

Response time is another major difference between an AI experiment and a usable business application.

Our practical target has been to keep AI responses around five seconds where possible. Within that time, the application may need to identify the user, determine accessible information, perform permission-aware vector retrieval, collect relevant chunks, construct the model context, call the language model and return the answer.

Improving only LLM inference time is therefore not enough. Database queries, permission filtering, vector search, context construction and network latency all contribute to the final user experience.

This is why database indexes and retrieval architecture became as important to us as the choice of AI model itself.

Enterprise RAG Does Not Have to Depend Entirely on Cloud AI

Most of our production experimentation has used managed AI infrastructure such as Amazon Bedrock, but we have also been testing smaller models locally.

For example, recent experiments with Qwen models on an office workstation equipped with 8 GB of VRAM produced useful results for certain workloads.

This does not mean a small local model is automatically a replacement for managed enterprise AI services. It does demonstrate that some AI workloads can increasingly be executed on modest local hardware.

For organizations with strict infrastructure, privacy or deployment requirements, hybrid and on-premise AI architectures are therefore becoming increasingly practical options worth evaluating.

RAG Is Only One Part of an Enterprise AI System

A production enterprise RAG application is much more than embeddings, vector search and an LLM.

It requires reliable integrations, document processing, synchronization, permissions, authentication, background jobs, database architecture, monitoring and application-level workflows.

In our experience, some of the most difficult engineering work happens outside the language model itself.

The real challenge is building a system where organizational knowledge can be continuously synchronized, efficiently retrieved and safely presented according to each user's existing access rights.

When those pieces work together, RAG becomes more than a document chatbot. It becomes an intelligent access layer across organizational knowledge.

Building Enterprise RAG at Web Foundation

At Web Foundation, we build AI-powered applications and RAG systems that connect language models with real business data and existing enterprise platforms.

Our work includes enterprise integrations, document processing, vector-search architecture, permission-aware retrieval, APIs, background processing and the surrounding software required to turn AI concepts into production applications.

Learn more about our AI & RAG Development Services or see how these concepts are applied in our Cortextual case study.