aboutsummaryrefslogtreecommitdiffstats
path: root/docs/versioned_docs/version-v0.27.0/07-development
diff options
context:
space:
mode:
authorMohamed Bassem <me@mbassem.com>2025-09-07 17:51:12 +0000
committerMohamed Bassem <me@mbassem.com>2025-09-07 17:51:12 +0000
commit13856a1fb33b1f2f07dddb32ac07db2bb0e82da3 (patch)
tree60d433efa07d2fa46485ecf9e99f7aa3ff72bd2e /docs/versioned_docs/version-v0.27.0/07-development
parent53f33d9975ae63498e64139efe41aafc8ea41d43 (diff)
downloadkarakeep-13856a1fb33b1f2f07dddb32ac07db2bb0e82da3.tar.zst
release(docs): release the 0.27 docs
Diffstat (limited to 'docs/versioned_docs/version-v0.27.0/07-development')
-rw-r--r--docs/versioned_docs/version-v0.27.0/07-development/01-setup.md178
-rw-r--r--docs/versioned_docs/version-v0.27.0/07-development/02-directories.md28
-rw-r--r--docs/versioned_docs/version-v0.27.0/07-development/03-database.md10
-rw-r--r--docs/versioned_docs/version-v0.27.0/07-development/04-architecture.md9
-rw-r--r--docs/versioned_docs/version-v0.27.0/07-development/_category_.json3
5 files changed, 228 insertions, 0 deletions
diff --git a/docs/versioned_docs/version-v0.27.0/07-development/01-setup.md b/docs/versioned_docs/version-v0.27.0/07-development/01-setup.md
new file mode 100644
index 00000000..6072a377
--- /dev/null
+++ b/docs/versioned_docs/version-v0.27.0/07-development/01-setup.md
@@ -0,0 +1,178 @@
+# Setup
+
+## Quick Start
+
+For the fastest way to get started with development, use the one-command setup script:
+
+```bash
+./start-dev.sh
+```
+
+This script will automatically:
+- Start Meilisearch in Docker (on port 7700)
+- Start headless Chrome in Docker (on port 9222)
+- Install dependencies with `pnpm install` if needed
+- Start both the web app and workers in parallel
+- Provide cleanup when you stop with Ctrl+C
+
+**Prerequisites:**
+- Docker installed and running
+- pnpm installed (see manual setup below for installation instructions)
+
+The script will output the running services:
+- Web app: http://localhost:3000
+- Meilisearch: http://localhost:7700
+- Chrome debugger: http://localhost:9222
+
+Press Ctrl+C to stop all services and clean up Docker containers.
+
+## Manual Setup
+
+Karakeep uses `node` version 22. To install it, you can use `nvm` [^1]
+
+```
+$ nvm install 22
+```
+
+Verify node version using this command:
+```
+$ node --version
+v22.14.0
+```
+
+Karakeep also makes use of `corepack`[^2]. If you have `node` installed, then `corepack` should already be
+installed on your machine, and you don't need to do anything. To verify the `corepack` is installed run:
+
+```
+$ command -v corepack
+/home/<user>/.nvm/versions/node/v22.14.0/bin/corepack
+```
+
+To enable `corepack` run the following command:
+
+```
+$ corepack enable
+```
+
+Then, from the root of the repository, install the packages and dependencies using:
+
+```
+$ pnpm install
+```
+
+Output of a successful `pnpm install` run should look something like:
+
+```
+Scope: all 20 workspace projects
+Lockfile is up to date, resolution step is skipped
+Packages: +3129
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+Progress: resolved 0, reused 2699, downloaded 0, added 3129, done
+
+devDependencies:
++ @karakeep/prettier-config 0.1.0 <- tooling/prettier
+
+. prepare$ husky
+└─ Done in 45ms
+Done in 5.5s
+```
+
+You can now continue with the rest of this documentation.
+
+### First Setup
+
+- You'll need to prepare the environment variables for the dev env.
+- Easiest would be to set it up once in the root of the repo and then symlink it in each app directory (e.g. `/apps/web`, `/apps/workers`) and also `/packages/db`.
+- Start by copying the template by `cp .env.sample .env`.
+- The most important env variables to set are:
+ - `DATA_DIR`: Where the database and assets will be stored. This is the only required env variable. You can use an absolute path so that all apps point to the same dir.
+ - `NEXTAUTH_SECRET`: Random string used to sign the JWT tokens. Generate one with `openssl rand -base64 36`. Logging in will not work if this is missing!
+ - `MEILI_ADDR`: If not set, search will be disabled. You can set it to `http://127.0.0.1:7700` if you run meilisearch using the command below.
+ - `OPENAI_API_KEY`: If you want to enable auto tag inference in the dev env.
+- run `pnpm run db:migrate` in the root of the repo to set up the database.
+
+### Dependencies
+
+#### Meilisearch
+
+Meilisearch is the provider for the full text search (and at some point embeddings search too). You can get it running with `docker run -p 7700:7700 getmeili/meilisearch:v1.13.3`.
+
+Mount persistent volume if you want to keep index data across restarts. You can trigger a re-index for the entire items collection in the admin panel in the web app.
+
+#### Chrome
+
+The worker app will automatically start headless chrome on startup for crawling pages. You don't need to do anything there.
+
+### Web App
+
+- Run `pnpm web` in the root of the repo.
+- Go to `http://localhost:3000`.
+
+> NOTE: The web app kinda works without any dependencies. However, search won't work unless meilisearch is running. Also, new items added won't get crawled/indexed unless workers are running.
+
+### Workers
+
+- Run `pnpm workers` in the root of the repo.
+
+### Mobile App (iOS & Android)
+
+#### Prerequisites
+
+To build and run the mobile app locally, you'll need:
+
+- **For iOS development**:
+ - macOS computer
+ - Xcode installed from the App Store
+ - iOS Simulator (comes with Xcode)
+
+- **For Android development**:
+ - Android Studio installed
+ - Android SDK configured
+ - Android Emulator or physical device
+
+For detailed setup instructions, refer to the [Expo documentation](https://docs.expo.dev/guides/local-app-development/).
+
+#### Running the app
+
+- `cd apps/mobile`
+- `pnpm exec expo prebuild --no-install` to build the app.
+
+**For iOS:**
+- `pnpm exec expo run:ios`
+- The app will be installed and started in the simulator.
+
+**Troubleshooting iOS Setup:**
+If you encounter an error like `xcrun: error: SDK "iphoneos" cannot be located`, you may need to set the correct Xcode developer directory:
+```bash
+sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
+```
+
+**For Android:**
+- Start the Android emulator or connect a physical device.
+- `pnpm exec expo run:android`
+- The app will be installed and started on the emulator/device.
+
+Changing the code will hot reload the app. However, installing new packages requires restarting the expo server.
+
+### Browser Extension
+
+- `cd apps/browser-extension`
+- `pnpm dev`
+- This will generate a `dist` package
+- Go to extension settings in chrome and enable developer mode.
+- Press `Load unpacked` and point it to the `dist` directory.
+- The plugin will pop up in the plugin list.
+
+In dev mode, opening and closing the plugin menu should reload the code.
+
+
+## Docker Dev Env
+
+If the manual setup is too much hassle for you. You can use a docker based dev environment by running `docker compose -f docker/docker-compose.dev.yml up` in the root of the repo. This setup wasn't super reliable for me though.
+
+
+[^1]: [nvm](https://github.com/nvm-sh/nvm) is a node version manager. You can install it following [these
+instructions](https://github.com/nvm-sh/nvm?tab=readme-ov-file#installing-and-updating).
+
+[^2]: [corepack](https://nodejs.org/api/corepack.html) is an experimental tool to help with managing versions of your
+package managers.
diff --git a/docs/versioned_docs/version-v0.27.0/07-development/02-directories.md b/docs/versioned_docs/version-v0.27.0/07-development/02-directories.md
new file mode 100644
index 00000000..712e86a1
--- /dev/null
+++ b/docs/versioned_docs/version-v0.27.0/07-development/02-directories.md
@@ -0,0 +1,28 @@
+# Directory Structure
+
+## Apps
+
+| Directory | Description |
+| ------------------------ | ------------------------------------------------------ |
+| `apps/web` | The main web app |
+| `apps/workers` | The background workers logic |
+| `apps/mobile` | The react native based mobile app |
+| `apps/browser-extension` | The browser extension |
+| `apps/landing` | The landing page of [karakeep.app](https://karakeep.app) |
+
+## Shared Packages
+
+| Directory | Description |
+| ----------------- | ---------------------------------------------------------------------------- |
+| `packages/db` | The database schema and migrations |
+| `packages/trpc` | Where most of the business logic lies built as TRPC routes |
+| `packages/shared` | Some shared code between the different apps (e.g. loggers, configs, assetdb) |
+
+## Toolings
+
+| Directory | Description |
+| -------------------- | ----------------------- |
+| `tooling/typescript` | The shared tsconfigs |
+| `tooling/eslint` | ESlint configs |
+| `tooling/prettier` | Prettier configs |
+| `tooling/tailwind` | Shared tailwind configs |
diff --git a/docs/versioned_docs/version-v0.27.0/07-development/03-database.md b/docs/versioned_docs/version-v0.27.0/07-development/03-database.md
new file mode 100644
index 00000000..d7a15b62
--- /dev/null
+++ b/docs/versioned_docs/version-v0.27.0/07-development/03-database.md
@@ -0,0 +1,10 @@
+# Database Migrations
+
+- The database schema lives in `packages/db/schema.ts`.
+- Changing the schema, requires a migration.
+- You can generate the migration by running `pnpm run db:generate --name description_of_schema_change` in the root dir.
+- You can then apply the migration by running `pnpm run db:migrate`.
+
+## Drizzle Studio
+
+You can start the drizzle studio by running `pnpm run db:studio` in the root of the repo.
diff --git a/docs/versioned_docs/version-v0.27.0/07-development/04-architecture.md b/docs/versioned_docs/version-v0.27.0/07-development/04-architecture.md
new file mode 100644
index 00000000..5a864034
--- /dev/null
+++ b/docs/versioned_docs/version-v0.27.0/07-development/04-architecture.md
@@ -0,0 +1,9 @@
+# Architecture
+
+![Architecture Diagram](/img/architecture/arch.png)
+
+- Webapp: NextJS based using sqlite for data storage.
+- Workers: Consume the jobs from sqlite based job queue and executes them, there are three job types:
+ 1. Crawling: Fetches the content of links using a headless chrome browser running in the workers container.
+ 2. OpenAI: Uses OpenAI APIs to infer the tags of the content.
+ 3. Indexing: Indexes the content in meilisearch for faster retrieval during search.
diff --git a/docs/versioned_docs/version-v0.27.0/07-development/_category_.json b/docs/versioned_docs/version-v0.27.0/07-development/_category_.json
new file mode 100644
index 00000000..73dd67ec
--- /dev/null
+++ b/docs/versioned_docs/version-v0.27.0/07-development/_category_.json
@@ -0,0 +1,3 @@
+{
+ "label": "Development",
+}