Fedora + Cloudflare Repository Platform

This repository houses the production infrastructure that powers Michael J Wright’s digital archive. Fedora 6 provides the preservation back-end, Cloudflare delivers the public experience, and the surrounding automation keeps the stack reliable for curators, developers, and partner institutions.

1. System Overview

Authoritative domains

• Public & documentation: https://data.michaeljwright.com.au
• Fedora tunnel endpoint: https://fcrepo.michaeljwright.com.au
• Branch alias (for internal previews): https://main.fedora-frontend.pages.dev
• Curators add and describe new works using secure proxy tools (no direct server access needed).
• Developers build integrations against the Worker-protected API endpoints.
• Partners retrieve syndicated exports for downstream collections and exhibitions.
• Visitors experience the public storytelling site at https://michaeljwright.com.au, which consumes the data curated here.

Stakeholder Roles

2. Quick Start

# Prerequisites: Docker Desktop + Compose, Node 18+, Cloudflare Wrangler CLI

git clone git@github.com:Rob142857/fedoraMJWArtist.git
cd fedoraMJWArtist

npm install                 # installs wrangler + build tooling
cp .env.example .env        # (optional) start from template; this repo may commit .env for VM deployment

# Runtime credentials live in config/runtime.env; Worker secrets are set via Wrangler.

pwsh -ExecutionPolicy Bypass -File .\quickstartdockerandcloudflared.ps1 -ForceRestartTunnel

The PowerShell script ensures Docker Desktop is running, starts the Compose stack, and verifies the Cloudflare Tunnel.

Collection Structure (Ready for Curation)

The repository is initialized with the following collection hierarchy:

/fcrepo/rest/
├── paintings/
│   ├── coastal-studies/
│   ├── desert-series/
│   ├── blue-mountains-landscapes/
│   ├── urban-fragments/
│   └── abstract-compositions/
├── drawings/
│   ├── sketches/
│   ├── prints/
│   └── mixed-media/
├── sculptures/
│   ├── bronze-works/
│   ├── wood-carvings/
│   └── installations/
├── photographs/
│   ├── nature-studies/
│   ├── native-flora/
│   ├── urban-fragments/
│   └── portrait-studies/
├── poems/
│   ├── winter-sonnets/
│   ├── nature-poetry/
│   └── urban-meditations/
└── notebooks/
    ├── outback-journals/
    ├── sketchbooks/
    └── travel-writing/

Access Points:

• Local curator access: http://localhost:8080/fcrepo/rest/
  • Username: (set in config/runtime.env as FEDORA_CURATOR_USERNAME)
  • Password: (set in config/runtime.env as FEDORA_CURATOR_PASSWORD)
• Public API: https://api.michaeljwright.com.au/fcrepo/rest/
• Direct tunnel: https://fcrepo.michaeljwright.com.au/fcrepo/rest/

For Curators: See docs/curators-guide.md for complete workflow documentation.

Deploy Worker & Pages

# Worker (API proxy)
npx wrangler secret put FEDORA_BASIC_AUTH   # paste "Basic base64(user:pass)" (use a dedicated non-admin/service user)
npx wrangler deploy

# Helper (recommended): prompts for username/password and sets the secret without manual base64 copy/paste
npm run secret:fedora:deploy

# Bootstrap (recommended for new setups): generates/rotates a dedicated Fedora service user in config/runtime.env,
# recreates the Fedora container (Tomcat picks up the new user), sets FEDORA_BASIC_AUTH, and deploys.
npm run bootstrap:syndication:creds

# Documentation / frontend
npm run pages:deploy

3. Repository & API Architecture

1. Collections & Metadata

   • Fedora containers mirror archival structure: Root → Collections → {Projects, Exhibitions, Writings}.
   • Descriptive metadata uses Dublin Core + custom JSON-LD; technical EXIF/IPTC kept as RDF or JSON binaries.
   • Fedora stores low-resolution, watermarked access copies (JPEG/WebP as needed). Full-resolution masters/originals are stored elsewhere and referenced from Fedora metadata.

2. Ingestion Workflow (Planned)

   • Manifest-driven uploader (CSV/JSON) posts binaries + metadata to Fedora REST.
   • Fedora memento keeps automatic version history.
   • WebAC or Worker enforcement separates curator, read-only, and public access paths.

3. Syndication APIs

   • Cloudflare Worker emits JSON (metadata + preview asset URLs).
   • Responses are cached with ETags and stored in CDN edge.
   • Search metadata will be mirrored into OpenSearch/Algolia for faceted discovery.

4. Public Site Integration

   • michaeljwright.com.au consumes Worker endpoints (/api/v1/...).
   • Public experiences use low-res, watermarked previews from the archive; master/original delivery is handled outside Fedora.
   • Editors access a secure Pages admin surface backed by the same API.

5. Partner Distribution

   • Nightly exports (JSON, CSV, METS) and optional OAI-PMH feed.
   • Webhooks / queues (e.g., Azure Service Bus) notify partners of new resources.

6. Governance & Operations

   • Rotate Worker secrets quarterly, enforce HTTPS only, audit Cloudflare tunnel logs.
   • Snapshot Fedora binaries + Postgres to offsite storage on a schedule.
   • Prometheus + Grafana dashboards track ingest latency, Worker health, tunnel status.
   • /docs captures metadata standards, runbooks, and onboarding guides.

4. Documentation Hub (/docs)

Run npm run build:docs to regenerate HTML artefacts from Markdown sources:

npm run pages:deploy rebuilds documentation and publishes to Cloudflare Pages (custom domain + branch alias).

Audience tip:

• Use the cards on the home page (/) for non-technical narratives, runbooks, and deployment checklists.
• /docs/README.html remains focused on technical operators; the upcoming Grace Notes page will translate the project vision for curators and stakeholders.
• /docs/metadata-schema.html provides a complete reference for all metadata fields including file technical metadata extracted at ingest.

5. Environment & Secrets

Docker runtime credentials

Fedora/Tomcat user credentials for the Docker host are read from:

• config/runtime.env (committed; private repo only)
• template: config/runtime.env.example

After updating config/runtime.env, apply it by recreating the Fedora container:

docker compose up -d --force-recreate mjw-fedora

Required entries in config/runtime.env (for syndication reads):

• FEDORA_CURATOR_USERNAME
• FEDORA_CURATOR_PASSWORD

Cloudflare Worker (api.data) required config

Secrets (Cloudflare-side):

• SYNDICATION_API_KEYS
• FEDORA_BASIC_AUTH (should match FEDORA_CURATOR_USERNAME/FEDORA_CURATOR_PASSWORD)

Vars (wrangler.toml / Cloudflare-side):

• FEDORA_URL
• CORS_ALLOW_ORIGINS (only needed for direct browser calls to /api/v1/*)
• RATE_LIMIT_PER_MINUTE

On Ubuntu (in this repo dir), after editing config/runtime.env you can sync FEDORA_BASIC_AUTH to Cloudflare with:

./scripts/sync-worker-fedora-basic-auth-from-runtime-env.sh
npx wrangler deploy --config wrangler.toml

On Ubuntu, a single-command bootstrap that updates the Tomcat service user password in config/runtime.env, recreates the Fedora container, syncs FEDORA_BASIC_AUTH, and deploys is available:

./scripts/bootstrap-syndication-creds.sh

If you intentionally want to rotate the Fedora service password (and then sync + deploy), run:

./scripts/bootstrap-syndication-creds.sh config/runtime.env wrangler.toml --rotate

Important: FEDORA_BASIC_AUTH must match the Fedora service user password currently active in Tomcat. Avoid rotating on one machine while syncing secrets from another.

After editing .env, rerun the quickstart script to restart containers and reload Tomcat config.

6. Key Commands

# Verify docker services & tunnel
pwsh -ExecutionPolicy Bypass -File .\quickstartdockerandcloudflared.ps1 -ForceRestartTunnel

# Tail Worker logs
npx wrangler tail

# Deploy Worker + documentation
npx wrangler deploy
npm run pages:preview   # preview on https://main.fedora-frontend.pages.dev
npm run pages:deploy    # publish to https://data.michaeljwright.com.au

# Compose operations
docker compose ps
docker compose logs fcrepo
docker compose down

7. Support & Next Steps

• Metadata schema finalisation (SHACL/JSON Schema).
• Bulk ingestion tooling & audit logging.
• Worker /api/v1 endpoints with pagination + search integration.
• Partner export feeds (JSON, CSV, METS, OAI-PMH) and IIIF support.
• Cloudflare Zero Trust for curator access and documentation portal.

OpenAPI

This repo includes an OpenAPI document describing the Worker endpoints used for integration and syndication:

• openapi.yaml

Live (syndication-only):

• OpenAPI JSON: https://api.michaeljwright.com.au/syndication/openapi.json
• Swagger UI: https://api.michaeljwright.com.au/syndication/swagger

It covers:

• manage.michaeljwright.com.au management/export APIs (/api/items, /api/export, /api/bulk-update)
• submit.michaeljwright.com.au ingest APIs (/api/submit, /api/validate-catalog-id)
• api.michaeljwright.com.au Fedora proxy surface (/fcrepo/rest/{path})

Syndication quickstart

Syndication is collaboration-only. If you want to consume/archive MJW materials in another site (e.g. a Vite/React frontend), contact AdminRob / project maintainers to set up access.

What partners must provide:

• Allowed origins: the exact production origin(s) that will call the API (e.g. https://your-site.pages.dev, https://yourdomain.com).
• Scope: which materials you need (collection(s) like paintings, photographs, etc.), and optional filters like series names or an explicit list of catalog IDs.
• Expected traffic: approximate peak requests/minute and typical page load pattern (initial load, infinite scroll, etc.).

What we issue/configure:

• API key for the read-only syndication endpoints (/api/v1/...) and strict CORS allowlist enforcement.
• Rate limit (best-effort in Worker + can be reinforced with Cloudflare rules if needed).

Security note: do not embed partner API keys in a public browser client; proxy requests through a server-side component (e.g., a Cloudflare Worker) so secrets stay private.

Consumer app (MJW-com-au): required secrets

For the Vite/React consumer site (“MJW-com-au”), the only credential required to use the syndication endpoints is:

• MJW_SYNDICATION_API_KEY (secret, stored server-side in the consumer project)

Recommended (non-secret) config:

• MJW_ARCHIVE_BASE_URL=https://api.michaeljwright.com.au

Implementation guidance (Pages Functions proxy pattern):

• docs/syndication-consumer-mjw-com-au.md

Track progress in docs/development-roadmap.md and update /docs on each sprint.