diff options
| -rw-r--r-- | README.md | 22 | ||||
| -rw-r--r-- | docs/docs/01-intro.md | 2 | ||||
| -rw-r--r-- | docs/docs/02-installation.md | 25 | ||||
| -rw-r--r-- | docs/docs/03-configuration.md | 1 | ||||
| -rw-r--r-- | docs/docs/04-screenshots.md | 26 | ||||
| -rw-r--r-- | docs/docs/05-quick-sharing.md (renamed from docs/docs/04-quick-sharing.md) | 0 | ||||
| -rw-r--r-- | docs/docs/06-openai.md (renamed from docs/docs/05-openai.md) | 0 | ||||
| -rw-r--r-- | docs/docs/07-Development/01-setup.md (renamed from docs/docs/06-Development/01-setup.md) | 0 | ||||
| -rw-r--r-- | docs/docs/07-Development/02-directories.md (renamed from docs/docs/06-Development/02-directories.md) | 0 | ||||
| -rw-r--r-- | docs/docs/07-Development/03-database.md (renamed from docs/docs/06-Development/03-database.md) | 0 | ||||
| -rw-r--r-- | docs/docs/07-Development/04-architecture.md | 10 | ||||
| -rw-r--r-- | docs/docs/08-security-considerations.md (renamed from docs/docs/07-security-considerations.md) | 0 | ||||
| -rw-r--r-- | docs/static/img/architecture/arch.png | bin | 0 -> 106144 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/admin.png | bin | 0 -> 237460 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/all-lists.png | bin | 0 -> 188073 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/all-tags.png | bin | 0 -> 278520 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/bookmark-preview.png | bin | 0 -> 2138263 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/homepage.png | bin | 0 -> 3196208 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/settings.png | bin | 0 -> 259583 bytes | |||
| -rw-r--r-- | docs/static/img/screenshots/share-sheet.png | bin | 0 -> 987806 bytes |
20 files changed, 59 insertions, 27 deletions
@@ -12,7 +12,7 @@ A self-hostable bookmark-everything app with a touch of AI for the data hoarders - 🔎 Full text search of all the content stored. - ✨ AI-based (aka chatgpt) automatic tagging. - 🔖 [Chrome plugin](https://chromewebstore.google.com/detail/hoarder/kgcjekpmcjjogibpjebkhaanilehneje) for quick bookmarking. -- 📱 [iOS shortcut](https://www.icloud.com/shortcuts/78734b46624c4a3297187c85eb50d800) for bookmarking content from the phone. A minimal mobile app is in the works. +- 📱 An iOS app that's pending apple's review. - 💾 Self-hosting first. - [Planned] Archiving the content for offline reading. @@ -22,6 +22,7 @@ A self-hostable bookmark-everything app with a touch of AI for the data hoarders - [Installation](https://docs.hoarder.app/installation) - [Configuration](https://docs.hoarder.app/configuration) +- [Screenshots](https://docs.hoarder.app/screenshots) - [Security Considerations](https://docs.hoarder.app/security-considerations) - [Development](https://docs.hoarder.app/Development/setup) @@ -53,11 +54,14 @@ I browse reddit, twitter and hackernews a lot from my phone. I frequently find i I'm a systems engineer in my day job (and have been for the past 7 years). I didn't want to get too detached from the web development world. I decided to build this app as a way to keep my hand dirty with web development, and at the same time, build something that I care about and will use everyday. -## Why not X? - -- [Pocket](getpocket.com): Pocket is what hooked me into the whole idea of read-it-later apps. I used it [a lot](https://blog.mbassem.com/2019/01/27/favorite-articles-2018/). However, I recently got into home-labbing and became obsessed with the idea of running my services in my home server. Hoarder is meant to be a self-hosting first app. -- [Omnivore](https://omnivore.app/): Omnivore is pretty cool open source read-it-later app. Unfortunately, it's heavily dependent on google cloud infra which makes self-hosting it quite hard. They published a [blog post](https://docs.omnivore.app/self-hosting/self-hosting.html) on how to run a minimal omnivore but it was lacking a lot of stuff. Self-hosting doesn't really seem to be a high priority for them, and that's something I care about, so I decided to build an alternative. -- [Instapaper](https://www.instapaper.com/): Not open source and not self-hostable. -- [memos](https://github.com/usememos/memos): I love memos. I have it running on my home server and it's one of my most used self-hosted apps. I, however, don't like the fact that it doesn't preview the content of the links I dump there and to be honest, it doesn't have to because that's not what it was designed for. It's just that I dump a lot of links there and I'd have loved if I'd be able to figure which link is that by just looking at my timeline. Also, given the variety of things I dump there, I'd have loved if it does some sort of automatic tagging for what I save there. This is exactly the usecase that I'm trying to tackle with Hoarder. -- [Wallabag](https://wallabag.it): Wallabag is a well-established open source read-it-later app written in php and I think it's the common recommendation on reddit for such apps. To be honest, I didn't give it a real shot, and the UI just felt a bit dated for my liking. Honestly, it's probably much more stable and feature complete than this app, but where's the fun in that? -- [Shiori](https://github.com/go-shiori/shiori): Shiori is meant to be an open source pocket clone written in Go. It ticks all the marks but doesn't have my super sophisticated AI-based tagging. (JK, I only found about it after I decided to build my own app, so here we are 🤷). +## Alternatives + +- [memos](https://github.com/usememos/memos): I love memos. I have it running on my home server and it's one of my most used self-hosted apps. It doesn't, however, archive or preview the links shared in it. It's just that I dump a lot of links there and I'd have loved if I'd be able to figure which link is that by just looking at my timeline. Also, given the variety of things I dump there, I'd have loved if it does some sort of automatic tagging for what I save there. This is exactly the usecase that I'm trying to tackle with Hoarder. +- [mymind](https://mymind.com/): Mymind is the closest alternative to this project and from where I drew a lot of inspirations. It's a commercial product though. +- [raindrop](https://raindrop.io): A polished open source bookmark manager that supports links, images and files. It's not self-hostable though. +- Bookmark managers (mostly focused on bookmarking links): + - [Pocket](getpocket.com): Pocket is what hooked me into the whole idea of read-it-later apps. I used it [a lot](https://blog.mbassem.com/2019/01/27/favorite-articles-2018/). However, I recently got into home-labbing and became obsessed with the idea of running my services in my home server. Hoarder is meant to be a self-hosting first app. + - [Linkwarden](https://linkwarden.app/): An open-source self-hostable bookmark manager that I ran for a bit in my homelab. It's focused mostly on links and supports collaborative collections. + - [Omnivore](https://omnivore.app/): Omnivore is pretty cool open source read-it-later app. Unfortunately, it's heavily dependent on google cloud infra which makes self-hosting it quite hard. They published a [blog post](https://docs.omnivore.app/self-hosting/self-hosting.html) on how to run a minimal omnivore but it was lacking a lot of stuff. Self-hosting doesn't really seem to be a high priority for them, and that's something I care about, so I decided to build an alternative. + - [Wallabag](https://wallabag.it): Wallabag is a well-established open source read-it-later app written in php and I think it's the common recommendation on reddit for such apps. To be honest, I didn't give it a real shot, and the UI just felt a bit dated for my liking. Honestly, it's probably much more stable and feature complete than this app, but where's the fun in that? + - [Shiori](https://github.com/go-shiori/shiori): Shiori is meant to be an open source pocket clone written in Go. It ticks all the marks but doesn't have my super sophisticated AI-based tagging. (JK, I only found about it after I decided to build my own app, so here we are 🤷). diff --git a/docs/docs/01-intro.md b/docs/docs/01-intro.md index 7418bbb6..4aae4b79 100644 --- a/docs/docs/01-intro.md +++ b/docs/docs/01-intro.md @@ -17,7 +17,7 @@ Hoarder is an open source "Bookmark Everything" app that uses AI for automatical - 🔎 Full text search of all the content stored. - ✨ AI-based (aka chatgpt) automatic tagging. - 🔖 [Chrome plugin](https://chromewebstore.google.com/detail/hoarder/kgcjekpmcjjogibpjebkhaanilehneje) for quick bookmarking. -- 📱 [iOS shortcut](https://www.icloud.com/shortcuts/78734b46624c4a3297187c85eb50d800) for bookmarking content from the phone. A minimal mobile app is in the works. +- 📱 An iOS app that's pending apple's review. - 💾 Self-hosting first. - [Planned] Archiving the content for offline reading. diff --git a/docs/docs/02-installation.md b/docs/docs/02-installation.md index 1ee01d79..0a25c7bf 100644 --- a/docs/docs/02-installation.md +++ b/docs/docs/02-installation.md @@ -16,37 +16,30 @@ Create a new directory to host the compose file and env variables. Download the docker compose file provided [here](https://github.com/MohamedBassem/hoarder-app/blob/main/docker/docker-compose.yml). ``` -$ wget https://raw.githubusercontent.com/MohamedBassem/hoarder-app/main/docker/docker-compose.yml +wget https://raw.githubusercontent.com/MohamedBassem/hoarder-app/main/docker/docker-compose.yml ``` ### 3. Populate the environment variables -You can use the env file template provided [here](https://github.com/MohamedBassem/hoarder-app/blob/main/.env.sample) and fill it manually using the documentation [here](/configuration). - -``` -$ wget https://raw.githubusercontent.com/MohamedBassem/hoarder-app/main/.env.sample -$ mv .env.sample .env -``` - -Alternatively, here is a minimal `.env` file to use: +To configure the app, create a `.env` file in the directory and add this minimal env file: ``` NEXTAUTH_SECRET=super_random_string -NEXTAUTH_URL=<YOUR DEPLOYED URL> - HOARDER_VERSION=release - -MEILI_ADDR=http://meilisearch:7700 MEILI_MASTER_KEY=another_random_string ``` -You can use `openssl rand -base64 36` to generate the random strings. +You **should** change the random strings. You can use `openssl rand -base64 36` to generate the random strings. Persistent storage and the wiring between the different services is already taken care of in the docker compose file. +Keep in mind that every time you change the `.env` file, you'll need to re-run `docker compose up`. + +If you want more config params, check the config documentation [here](/configuration). + ### 4. Setup OpenAI -To enable automatic tagging, you'll need to configure open ai. This is optional though but hightly recommended. +To enable automatic tagging, you'll need to configure OpenAI. This is optional though but hightly recommended. - Follow [OpenAI's help](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) to get an API key. - Add `OPENAI_API_KEY=<key>` to the env file. @@ -60,5 +53,5 @@ Learn more about the costs of using openai [here](/openai). Start the service by running: ``` -$ docker compose up -d +docker compose up -d ``` diff --git a/docs/docs/03-configuration.md b/docs/docs/03-configuration.md index a9c02611..585d25b5 100644 --- a/docs/docs/03-configuration.md +++ b/docs/docs/03-configuration.md @@ -6,7 +6,6 @@ The app is mainly configured by environment variables. All the used environment | ---------------- | ------------------------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------- | | DATA_DIR | Yes | Not set | The path for the persistent data directory. This is where the db and the uploaded assets live. | | NEXTAUTH_SECRET | Yes | Not set | Random string used to sign the JWT tokens. Generate one with `openssl rand -base64 36`. | -| NEXTAUTH_URL | Yes | Not set | The url on which the service will be running on. E.g. (`https://demo.hoarder.app`). | | REDIS_HOST | Yes | localhost | The address of redis used by background jobs | | REDIS_POST | Yes | 6379 | The port of redis used by background jobs | | OPENAI_API_KEY | No | Not set | The OpenAI key used for automatic tagging. If not set, automatic tagging won't be enabled. More on that in [here](/openai). | diff --git a/docs/docs/04-screenshots.md b/docs/docs/04-screenshots.md new file mode 100644 index 00000000..03137367 --- /dev/null +++ b/docs/docs/04-screenshots.md @@ -0,0 +1,26 @@ +# Screenshots + +## Homepage + + + +## Tags + + + +## Lists + + + +## Settings + + + +## Admin Panel + + + + +## iOS Sharing + +<img src="/img/screenshots/share-sheet.png" width="400px" /> diff --git a/docs/docs/04-quick-sharing.md b/docs/docs/05-quick-sharing.md index 05ff5448..05ff5448 100644 --- a/docs/docs/04-quick-sharing.md +++ b/docs/docs/05-quick-sharing.md diff --git a/docs/docs/05-openai.md b/docs/docs/06-openai.md index 91e37c07..91e37c07 100644 --- a/docs/docs/05-openai.md +++ b/docs/docs/06-openai.md diff --git a/docs/docs/06-Development/01-setup.md b/docs/docs/07-Development/01-setup.md index 775a5806..775a5806 100644 --- a/docs/docs/06-Development/01-setup.md +++ b/docs/docs/07-Development/01-setup.md diff --git a/docs/docs/06-Development/02-directories.md b/docs/docs/07-Development/02-directories.md index 54552402..54552402 100644 --- a/docs/docs/06-Development/02-directories.md +++ b/docs/docs/07-Development/02-directories.md diff --git a/docs/docs/06-Development/03-database.md b/docs/docs/07-Development/03-database.md index 40e2d164..40e2d164 100644 --- a/docs/docs/06-Development/03-database.md +++ b/docs/docs/07-Development/03-database.md diff --git a/docs/docs/07-Development/04-architecture.md b/docs/docs/07-Development/04-architecture.md new file mode 100644 index 00000000..df69376a --- /dev/null +++ b/docs/docs/07-Development/04-architecture.md @@ -0,0 +1,10 @@ +# Architecture + + + +- Webapp: NextJS based using sqlite for data storage. +- Redis: Used with BullMQ for scheduling background jobs for the workers. +- Workers: Consume the jobs from redis and executes them, there are three job types: + 1. Crawling: Fetches the content of links using a headless chrome browser running in the workers container. + 2. OpenAI: Uses OpenAI APIs to infer the tags of the content. + 3. Indexing: Indexes the content in meilisearch for faster retrieval during search. diff --git a/docs/docs/07-security-considerations.md b/docs/docs/08-security-considerations.md index 7cab2e07..7cab2e07 100644 --- a/docs/docs/07-security-considerations.md +++ b/docs/docs/08-security-considerations.md diff --git a/docs/static/img/architecture/arch.png b/docs/static/img/architecture/arch.png Binary files differnew file mode 100644 index 00000000..7583970e --- /dev/null +++ b/docs/static/img/architecture/arch.png diff --git a/docs/static/img/screenshots/admin.png b/docs/static/img/screenshots/admin.png Binary files differnew file mode 100644 index 00000000..cbc0cd86 --- /dev/null +++ b/docs/static/img/screenshots/admin.png diff --git a/docs/static/img/screenshots/all-lists.png b/docs/static/img/screenshots/all-lists.png Binary files differnew file mode 100644 index 00000000..befecb4b --- /dev/null +++ b/docs/static/img/screenshots/all-lists.png diff --git a/docs/static/img/screenshots/all-tags.png b/docs/static/img/screenshots/all-tags.png Binary files differnew file mode 100644 index 00000000..669ba5a4 --- /dev/null +++ b/docs/static/img/screenshots/all-tags.png diff --git a/docs/static/img/screenshots/bookmark-preview.png b/docs/static/img/screenshots/bookmark-preview.png Binary files differnew file mode 100644 index 00000000..200c598a --- /dev/null +++ b/docs/static/img/screenshots/bookmark-preview.png diff --git a/docs/static/img/screenshots/homepage.png b/docs/static/img/screenshots/homepage.png Binary files differnew file mode 100644 index 00000000..d4403659 --- /dev/null +++ b/docs/static/img/screenshots/homepage.png diff --git a/docs/static/img/screenshots/settings.png b/docs/static/img/screenshots/settings.png Binary files differnew file mode 100644 index 00000000..ed59ae33 --- /dev/null +++ b/docs/static/img/screenshots/settings.png diff --git a/docs/static/img/screenshots/share-sheet.png b/docs/static/img/screenshots/share-sheet.png Binary files differnew file mode 100644 index 00000000..538b9540 --- /dev/null +++ b/docs/static/img/screenshots/share-sheet.png |
