Overview
Manually extracting and indexing documents across formats like text, PDF, and JSON is tedious and error-prone. This workflow automates reading files from Google Drive, extracting text intelligently, splitting it into context-aware chunks, generating embeddings via OpenAI, and storing them in PostgreSQL—enabling seamless semantic search and retrieval-augmented generation (RAG) without manual overhead.
The Impact
- Slash manual processing time. Automate multi-format document extraction and vectorization end-to-end.
- Guarantee uniform indexing. Convert diverse file types into consistent, searchable vector data.
- Enable lightning-fast retrieval. Store vectors in PostgreSQL for efficient semantic queries.
- Support RAG applications. Prepare ready-to-use vector corpora for retrieval-augmented generation workflows.
Who This Is For
- Knowledge Managers seeking to bulk-vectorize enterprise documents for semantic Q&A.
- Product Managers needing rapid access to technical specs and whitepapers via vector search.
- Developers preparing large corpora for RAG system backends.
- Data Engineers implementing unified indexing pipelines for heterogeneous document formats.
How It Works
- Read File from Google Drive
- Fetch raw bytes or text content of the specified file using its FileId and Drive credentials.
- Determine File Type & Extract Text
- Automatically branch by file type (text/pdf/json) to extract clean, unified text content.
- Split Text Into Chunks
- Segment text into overlapping chunks based on configurable size and overlap for context retention.
- Generate Embeddings
- Call OpenAI's embeddings API with chunked texts using provided API key and optional custom endpoint.
- Format & Store Vectors
- Align embeddings with chunks and write vectors plus metadata into PostgreSQL under the specified collection.
What You'll Need
Before using this template, make sure you have:
- Google Drive file(s) in supported formats (text, PDF, JSON) and their FileIds.Google Drive API credentials authorized to read those files.PostgreSQL database with a prepared table (e.g., collection_vectors) to store embeddings, content, metadata, and collection name.OpenAI API key for embeddings generation and optional custom OpenAI API base URL.
How to Use
- Step 1. Prepare Your Files and Database
- Step 2. Configure Credentials
- Step 3. Enter Parameters
- Step 4. Run the Workflow
- Step 5. Verify Results
Ensure your Google Drive files are ready and your PostgreSQL vector table is set up with required fields.
Set up Google Drive API credentials in the workflow to enable file reading.
Input FileId, FileType (optional), OpenAI API key, PostgreSQL connection string, collection name, and chunking parameters.
Trigger execution to automatically extract, vectorize, and store document embeddings.
Check the workflow output for success status and inserted vector count to confirm completion.
