Step-by-Step Tutorial: Ingesting Alfresco Content Services into PGVector with Zero-Trust ACLs, Audit Reports & MCP Inspector

01.Prerequisites & Tutorial Workspace Bundle

Everything required for this walkthrough is bundled under the repository's tutorials/ folder (including a dedicated Docker Compose stack, sample documents across four permission tiers, JSON connector/job templates, MCP Inspector config files, and automation scripts).

• Docker Engine 24+ & Compose v2 (docker compose) with at least 8 GB RAM allocated (12 GB recommended for running Alfresco, Ollama, Kafka, and PGVector concurrently).
• Node.js 18+ (with npm / npx) to launch @modelcontextprotocol/inspector.
• curl and jq CLI utilities.

git clone https://github.com/opencrawling/opencrawling.git
cd opencrawling
ls -1 tutorials/
# config/   data/   docker/   mcp/   scripts/   README.md

02.Start the Local Environment with Docker

Launch the self-contained stack defined in tutorials/docker/docker-compose-alfresco-pgvector-mcp.yml. It brings up Alfresco Community 26.1.0 (running with -Dindex.subsystem.name=noindex for fast, low-memory startup in Folder Traversal mode), Alfresco Share, Alfresco Content App (the open-source Community counterpart of Digital Workspace) behind a Traefik reverse proxy on port 8081, PostgreSQL + pgvector, Redis Stack, Ollama (automatically pulling mxbai-embed-large), Apache Kafka, all three decoupled OpenCrawling consumers, the Secure MCP Server, and the React Admin UI:

./tutorials/scripts/start-env.sh

03.Prepare Test Documents with Zero-Trust Permissions in Alfresco

Now populate Alfresco with three test users (alice, bob, charlie), two groups (GROUP_engineering containing alice and bob; GROUP_finance containing alice and charlie), and four documents under /opencrawling-tutorial. Each folder disables permission inheritance so we can verify public access, group ACLs, broken inheritance, and explicit deny precedence:

./tutorials/scripts/provision-alfresco-data.sh

Under the hood, the provisioning script invokes Alfresco REST API v1 endpoints (POST /people, POST /groups, POST /groups/{id}/members, POST /nodes/{parentId}/children, PUT /nodes/{nodeId}/content, and PUT /nodes/{nodeId}?include=permissions) and saves the resulting Alfresco Node UUIDs to tutorials/data/.provisioned-nodes.json.

Inspect the Permission Schema Visually in Alfresco Share & Content App (Workspace)

Before running the ingestion job, open either Alfresco Share (http://localhost:8081/share/) or Alfresco Content App / Workspace (http://localhost:8081/workspace/) to explore the provisioned folder hierarchy and verify how Alfresco enforces these ACLs visually:

• Inspect Inherited & Locally Set ACLs as admin: Sign in with admin / admin, click Repository, and open opencrawling-tutorial. In Share, hover over any folder or file and choose Manage Permissions; in Content App / Workspace, select a node and open More actions (…) → Permissions to see permission inheritance disabled on each tier folder alongside the local authority assignments.
• Verify Persona Visibility (alice, bob, charlie): Sign out and log back in as each test user with password Password123!, then browse Repository → opencrawling-tutorial:
  • alice sees all four folders (public, finance, engineering, executive) and all four documents.
  • charlie sees only public and finance (with q4-financial-forecast.txt).
  • bob sees public and the engineering folder, yet opening engineering/ displays an empty folder because his explicit DENIED ACE on zero-trust-architecture.txt overrides his GROUP_engineering membership.

04.Configure Connectors & Execute the Ingestion Job

Open the OpenCrawling Admin UI at http://localhost:3000 (or automate the exact same REST calls with ./tutorials/scripts/configure-opencrawling.sh):

1. Check Pre-Seeded Connectors: Under Connectors, notice Ollama_Embedding_Default (mxbai-embed-large on the Transformation tab) and PGVector_Output (on the Output tab) are pre-registered.
2. Create the Alfresco Repository Connector: Switch to the Repository tab, click New Connector, set Connector Name to Alfresco_Tutorial, Connector Class to Alfresco Content Services Repository, Alfresco API URL to http://alfresco:8080/alfresco/api/-default-/public/alfresco/versions/1, Username / Password to admin / admin, Crawl Mode to Folder Tree Traversal, and Root Folder Path to /opencrawling-tutorial. Keep Include Subfolders Recursively, Fetch Binary Content Stream, and Include Access Control Lists (ACLs) & Collocated Security checked. Click Check Connection to see Connection Verified & Reliable, then click Save.
3. Create the Ingestion Job: Open Job Management → New Pipeline Job. Set Job Name to Alfresco_Tutorial_PGVector_Job, Repository Connector to Alfresco_Tutorial, Output Connector to PGVector_Output, Authority Connector to None (Public Access) (since collocated ACLs are enabled directly on the Alfresco connector), Path to /opencrawling-tutorial, Transformation Connector to Ollama_Embedding_Default, and Pipeline Mode to RAG Mode (Default). Save the job.
4. Start the Crawl: Click the play button (Start crawl job) next to Alfresco_Tutorial_PGVector_Job and follow progress in Real-time Logs until status becomes Finished (4 documents).

# Inspect indexed chunks and ACL metadata stored in vector_store_1024
docker exec -i postgres-vector-tutorial psql -U opencrawling -d opencrawling -c "
SELECT metadata->>'name' AS file_name,
       metadata->'security_read_keys' AS read_keys,
       metadata->'security_denied_read' AS denied_read
FROM vector_store_1024
ORDER BY file_name;"

05.Inspect Job Execution in Reports & Audit (Admin UI + DuckDB)

In the Admin UI left sidebar, click Reports & Audit to open the Reports & Audit Ledger:

• Execution Summary & KPIs: Verify Alfresco_Tutorial_PGVector_Job shows status SUCCESS, 4 total documents, 4 successful, 0 failed, 4 chunks, and 400 tokens.
• Document Ledger & Compliance Exports: Select the execution row to inspect per-document durations and Claim-Check URIs, or download the CSV Manifest, Hive-partitioned ZSTD Parquet File, and JSON Summary.
• Query with DuckDB: Click Query with DuckDB to open the Serverless Parquet SQL Playground (DuckDB) and click Execute Query:

SELECT document_id,
       source_uri,
       status,
       duration_ms,
       rag_chunk_count,
       rag_token_count
FROM read_parquet('/data/reports/**/*.parquet')
ORDER BY document_id;

06.Install, Configure & Query with MCP Inspector

Next, connect Anthropic's official MCP Inspector to OpenCrawling's Streamable HTTP MCP Server to inspect tools, test permission enforcement, and run queries as each persona.

Install & Launch MCP Inspector

With Node.js 18+ installed, run MCP Inspector directly with npx (or install globally via npm install -g @modelcontextprotocol/inspector), pointing it at tutorials/mcp/mcp-inspector-config.json:

npx @modelcontextprotocol/inspector \
  --config tutorials/mcp/mcp-inspector-config.json \
  --server opencrawling-alfresco

In the MCP Inspector browser UI (typically opened automatically on http://localhost:6274 with its session token), verify in the left panel that Transport Type is Streamable HTTP and URL is http://localhost:8080/mcp, then click Connect. Open the Tools tab and click List Tools to view all 11 tools (secureVectorSearch, getDocumentContent, listAccessibleSources, openarch_search, openarch_fetch_document, listRepositoryConnectors, exploreRepositoryTree, queryRepositoryDryRun, fetchJobTraces, getErrorLogs, and queryThroughputMetrics).

Run Permission-Enforced Queries Across Personas

Copy payloads from tutorials/mcp/sample-queries.json into the MCP Inspector Tools panel:

1. Public Access Only (guest): Call listAccessibleSources with {"userPrincipal":"guest","userRoles":"","dimensions":1024} → returns only company-handbook.txt (acl: "public"). Pass "dimensions": 1024 so only vector_store_1024 is enumerated.
2. Finance Group Access (charlie): Call secureVectorSearch with {"query":"Q4 operating budget financial forecast","userPrincipal":"charlie","userRoles":"GROUP_finance","maxResults":10,"minScore":0.0} → returns q4-financial-forecast.txt (notice Alfresco's GROUP_ prefix!) while excluding engineering and executive records.
3. Explicit Deny Precedence (bob): Call secureVectorSearch with {"query":"zero-trust architecture rotating mTLS certificates","userPrincipal":"bob","userRoles":"GROUP_engineering","maxResults":10,"minScore":0.0} → even though bob belongs to GROUP_engineering, zero-trust-architecture.txt is blocked and omitted. Calling getDocumentContent directly with the UUID of board-resolution-2026.txt fails closed with Document not found or access denied.
4. Full Executive + Engineering Access (alice): Call secureVectorSearch with {"query":"Project Horizon board resolution M&A strategy","userPrincipal":"alice","userRoles":"GROUP_engineering,GROUP_finance","maxResults":5,"minScore":0.0} → retrieves board-resolution-2026.txt, and getDocumentContent reveals its full unredacted text.
5. Live Repository Discovery Check: Call exploreRepositoryTree with {"connector":"Alfresco_Tutorial","userPrincipal":"bob","userRoles":"GROUP_engineering"} vs alice → bob sees only folders ["engineering","public"] and ["company-handbook.txt"], whereas alice sees all 4 folders and all 4 files directly from Alfresco without downloading binaries.

Run Automated End-to-End CLI Verification

You can also execute all PGVector SQL, Reports/DuckDB, and MCP Inspector CLI assertions in one command, and tear down when done:

# Verify PGVector ACL metadata, Reports API + DuckDB SQL, and all 4 MCP personas
./tutorials/scripts/verify-tutorial.sh

# Stop containers (pass --clean to remove volumes and provisioned state)
./tutorials/scripts/stop-env.sh