diff options
| author | Mohamed Bassem <me@mbassem.com> | 2025-02-02 17:42:36 +0000 |
|---|---|---|
| committer | Mohamed Bassem <me@mbassem.com> | 2025-02-02 17:42:36 +0000 |
| commit | f91fe67c9e881b7626deade95cf7cc9893cae137 (patch) | |
| tree | 4d0457c5c82acf5bb6b69fe3edc7354b97b98025 /docs/versioned_docs/version-v0.22.0/07-Development | |
| parent | ed8fd793f2c3ee0135c9fa230af7ead867a614bc (diff) | |
| download | karakeep-f91fe67c9e881b7626deade95cf7cc9893cae137.tar.zst | |
docs: Release the 0.22 docs
Diffstat (limited to 'docs/versioned_docs/version-v0.22.0/07-Development')
4 files changed, 111 insertions, 0 deletions
diff --git a/docs/versioned_docs/version-v0.22.0/07-Development/01-setup.md b/docs/versioned_docs/version-v0.22.0/07-Development/01-setup.md new file mode 100644 index 00000000..6d359234 --- /dev/null +++ b/docs/versioned_docs/version-v0.22.0/07-Development/01-setup.md @@ -0,0 +1,63 @@ +# Setup + +## Manual Setup +### First Setup + +- You'll need to prepare the environment variables for the dev env. +- Easiest would be to set it up once in the root of the repo and then symlink it in each app directory (e.g. `/apps/web`, `/apps/workers`) and also `/packages/db`. +- Start by copying the template by `cp .env.sample .env`. +- The most important env variables to set are: + - `DATA_DIR`: Where the database and assets will be stored. This is the only required env variable. You can use an absolute path so that all apps point to the same dir. + - `NEXTAUTH_SECRET`: Random string used to sign the JWT tokens. Generate one with `openssl rand -base64 36`. Logging in will not work if this is missing! + - `MEILI_ADDR`: If not set, search will be disabled. You can set it to `http://127.0.0.1:7700` if you run meilisearch using the command below. + - `OPENAI_API_KEY`: If you want to enable auto tag inference in the dev env. +- run `pnpm run db:migrate` in the root of the repo to set up the database. + +### Dependencies + +#### Meilisearch + +Meilisearch is the provider for the full text search. You can get it running with `docker run -p 7700:7700 getmeili/meilisearch:v1.11.1`. + +Mount persistent volume if you want to keep index data across restarts. You can trigger a re-index for the entire items collection in the admin panel in the web app. + +#### Chrome + +The worker app will automatically start headless chrome on startup for crawling pages. You don't need to do anything there. + +### Web App + +- Run `pnpm web` in the root of the repo. +- Go to `http://localhost:3000`. + +> NOTE: The web app kinda works without any dependencies. However, search won't work unless meilisearch is running. Also, new items added won't get crawled/indexed unless workers are running. + +### Workers + +- Run `pnpm workers` in the root of the repo. + +### iOS Mobile App + +- `cd apps/mobile` +- `pnpm exec expo prebuild --no-install` to build the app. +- Start the ios simulator. +- `pnpm exec expo run:ios` +- The app will be installed and started in the simulator. + +Changing the code will hot reload the app. However, installing new packages requires restarting the expo server. + +### Browser Extension + +- `cd apps/browser-extension` +- `pnpm dev` +- This will generate a `dist` package +- Go to extension settings in chrome and enable developer mode. +- Press `Load unpacked` and point it to the `dist` directory. +- The plugin will pop up in the plugin list. + +In dev mode, opening and closing the plugin menu should reload the code. + + +## Docker Dev Env + +If the manual setup is too much hassle for you. You can use a docker based dev environment by running `docker compose -f docker/docker-compose.dev.yml up` in the root of the repo. This setup wasn't super reliable for me though. diff --git a/docs/versioned_docs/version-v0.22.0/07-Development/02-directories.md b/docs/versioned_docs/version-v0.22.0/07-Development/02-directories.md new file mode 100644 index 00000000..54552402 --- /dev/null +++ b/docs/versioned_docs/version-v0.22.0/07-Development/02-directories.md @@ -0,0 +1,28 @@ +# Directory Structure + +## Apps + +| Directory | Description | +| ------------------------ | ------------------------------------------------------ | +| `apps/web` | The main web app | +| `apps/workers` | The background workers logic | +| `apps/mobile` | The react native based mobile app | +| `apps/browser-extension` | The browser extension | +| `apps/landing` | The landing page of [hoarder.app](https://hoarder.app) | + +## Shared Packages + +| Directory | Description | +| ----------------- | ---------------------------------------------------------------------------- | +| `packages/db` | The database schema and migrations | +| `packages/trpc` | Where most of the business logic lies built as TRPC routes | +| `packages/shared` | Some shared code between the different apps (e.g. loggers, configs, assetdb) | + +## Toolings + +| Directory | Description | +| -------------------- | ----------------------- | +| `tooling/typescript` | The shared tsconfigs | +| `tooling/eslint` | ESlint configs | +| `tooling/prettier` | Prettier configs | +| `tooling/tailwind` | Shared tailwind configs | diff --git a/docs/versioned_docs/version-v0.22.0/07-Development/03-database.md b/docs/versioned_docs/version-v0.22.0/07-Development/03-database.md new file mode 100644 index 00000000..40e2d164 --- /dev/null +++ b/docs/versioned_docs/version-v0.22.0/07-Development/03-database.md @@ -0,0 +1,11 @@ +# Database Migrations + +- The database schema lives in `packages/db/schema.ts`. +- Changing the schema, requires a migration. +- You can generate the migration by running `pnpm drizzle-kit generate:sqlite` in the `packages/db` dir. +- You can then apply the migration by running `pnpm run migrate`. + + +## Drizzle Studio + +You can start the drizzle studio by running `pnpm db:studio` in the root of the repo. diff --git a/docs/versioned_docs/version-v0.22.0/07-Development/04-architecture.md b/docs/versioned_docs/version-v0.22.0/07-Development/04-architecture.md new file mode 100644 index 00000000..5a864034 --- /dev/null +++ b/docs/versioned_docs/version-v0.22.0/07-Development/04-architecture.md @@ -0,0 +1,9 @@ +# Architecture + + + +- Webapp: NextJS based using sqlite for data storage. +- Workers: Consume the jobs from sqlite based job queue and executes them, there are three job types: + 1. Crawling: Fetches the content of links using a headless chrome browser running in the workers container. + 2. OpenAI: Uses OpenAI APIs to infer the tags of the content. + 3. Indexing: Indexes the content in meilisearch for faster retrieval during search. |
