| Commit message (Collapse) | Author | Age | Files | Lines | ||
|---|---|---|---|---|---|---|
| ... | ||||||
| * | feature(web): Add ability to manually trigger full page archives. Fixes #398 ↵ | kamtschatka | 2024-09-30 | 1 | -3/+5 | |
| | | | | | | | | | | | | | | (#418) * [Feature Request] Ability to select what to "crawl full page archive" #398 Added the ability to start a full page crawl for links and also in bulk operations added the ability to refresh links as a bulk operation as well * minor icon and wording changes --------- Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | feature(web): Add the ability to customize the inference prompts. Fixes #170 | MohamedBassem | 2024-09-29 | 1 | -39/+42 | |
| | | ||||||
| * | fix(workers): Log stacktrace on worker error. #424 (#429) | kamtschatka | 2024-09-26 | 3 | -3/+7 | |
| | | | | extended logging when an exception occurrs, so it is possible to see the stacktrace of a failed execution | |||||
| * | deps: Upgrade drizzle and next auth drizzle adapter | MohamedBassem | 2024-09-15 | 1 | -1/+1 | |
| | | ||||||
| * | feature(worker): Allow configuring inference job timeout and ollama keep ↵ | MohamedBassem | 2024-09-15 | 2 | -1/+2 | |
| | | | | | alive. Fixes #389 #224 | |||||
| * | build: Fix sherif failures by sorting deps | MohamedBassem | 2024-08-31 | 1 | -1/+1 | |
| | | ||||||
| * | fix(workers): Shutdown workers on SIGTERM | MohamedBassem | 2024-07-28 | 2 | -0/+9 | |
| | | ||||||
| * | fix: async/await issues with the new queue (#319) | kamtschatka | 2024-07-21 | 2 | -3/+3 | |
| | | ||||||
| * | refactor: Replace the usage of bullMQ with the hoarder sqlite-based queue (#309) | Mohamed Bassem | 2024-07-21 | 5 | -72/+75 | |
| | | ||||||
| * | fix: monolith not embedding SVG files correctly. Fixes #289 (#306) | kamtschatka | 2024-07-14 | 1 | -5/+2 | |
| | | | | passing in the URL of the page to have the proper URL for resolving relative paths | |||||
| * | refactor: added the bookmark type to the database (#256) | kamtschatka | 2024-07-01 | 1 | -0/+6 | |
| | | | | | | | | | | | | | | | | | | * refactoring asset types Extracted out functions to silently delete assets and to update them after crawling Generalized the mapping of assets to bookmark fields to make extending them easier * Added the bookmark type to the database Introduced an enum to have better type safety cleaned up the code and based some code on the type directly * add BookmarkType.UNKNWON * lint and remove unused function --------- Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | refactor: remove redundant code from crawler worker and refactor handling of ↵ | kamtschatka | 2024-06-29 | 1 | -32/+49 | |
| | | | | | | | | | | | | | | asset types (#253) * refactoring asset types Extracted out functions to silently delete assets and to update them after crawling Generalized the mapping of assets to bookmark fields to make extending them easier * revert silentDeleteAsset and hide better-sqlite3 --------- Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | feature: Automatically transfer image urls into bookmared assets. Fixes #246 | MohamedBassem | 2024-06-23 | 1 | -6/+16 | |
| | | ||||||
| * | refactor: extract assets into their own database table. #215 (#220) | kamtschatka | 2024-06-23 | 1 | -29/+71 | |
| | | | | | | | | | | | | | | | | | | | | * Allow downloading more content from a webpage and index it #215 added a new table that contains the information about assets for link bookmarks created migration code that transfers the existing data into the new table * Allow downloading more content from a webpage and index it #215 removed the old asset columns from the database updated the UI to use the data from the linkBookmarkAssets array * generalize the assets table to not be linked in particular to links * fix migrations post merge * fix missing asset ids in the getBookmarks call --------- Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | feature: add support for PDF links. Fixes #28 (#216) | kamtschatka | 2024-06-22 | 1 | -57/+163 | |
| | | | | | | | | | | | | | | | | | | * feature request: pdf support #28 Added a new sourceUrl column to the asset bookmarks Added transforming a link bookmark pointing at a pdf to an asset bookmark made sure the "View Original" link is also shown for asset bookmarks that have a sourceURL updated gitignore for IDEA * remove pdf parsing from the crawler * extract the http logic into its own function to avoid duplicating the post-processing actions (openai/index) * Add 5s timeout to the content type fetch --------- Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | fix: Trigger search re-index on bookmark tag manual updates. Fixes #208 (#210) | kamtschatka | 2024-06-09 | 2 | -10/+4 | |
| | | | | | | | | | | | | | * re-index of database is not scanning all places when bookmark tags are changed. Manual indexing is working as workaround #208 introduced a new function to trigger a reindex to reduce copy/paste added missing reindexes when tags are deleted/bookmarks are updated * give functions a bit more descriptive name --------- Co-authored-by: kamtschatka <simon.schatka@gmx.at> Co-authored-by: MohamedBassem <me@mbassem.com> | |||||
| * | fix(workers): AI infered tags can contain " " at the beginning. Fixes #184 ↵ | kamtschatka | 2024-06-07 | 1 | -3/+5 | |
| | | | | | | | | (#194) added a trim to tags to prevent whitespaces at the beginning/end of tags Co-authored-by: kamtschatka <simon.schatka@gmx.at> | |||||
| * | fix(crawler): Only update the database if full page archival is enabled | MohamedBassem | 2024-05-26 | 1 | -19/+19 | |
| | | ||||||
| * | feature: Full page archival with monolith. Fixes #132 | MohamedBassem | 2024-05-26 | 2 | -1/+66 | |
| | | ||||||
| * | feature(inference): Improve ollama tagging (#162) | kamtschatka | 2024-05-18 | 1 | -5/+12 | |
| | | | | | | | | | | | | | | * Inference Failed with Ollama #20 Changed the prompt to be split in 2, so ollama does not forget them * Update apps/workers/openaiWorker.ts Co-authored-by: Mohamed Bassem <me@mbassem.com> --------- Co-authored-by: kamtschatka <simon.schatka@gmx.at> Co-authored-by: Mohamed Bassem <me@mbassem.com> | |||||
| * | feature(crawler): Allow connecting to browser's websocket address and ↵ | MohamedBassem | 2024-05-15 | 1 | -28/+55 | |
| | | | | | launching the browser on demand. This enables support for browserless | |||||
| * | feature: Take full page screenshots #143 (#148) | kamtschatka | 2024-05-12 | 1 | -1/+2 | |
| | | | | | | | Added the fullPage flag to take full screen screenshots updated the UI accordingly to properly show the screenshots instead of scaling it down Co-authored-by: kamtschatka <simon.schatka@gmx.at> | |||||
| * | fix(inference): Attempt to reuse existing identical tags | MohamedBassem | 2024-04-26 | 1 | -22/+62 | |
| | | ||||||
| * | feature(crawler): Allow increasing crawler concurrency and configure storing ↵ | MohamedBassem | 2024-04-26 | 1 | -0/+13 | |
| | | | | | images and screenshots | |||||
| * | fix(crawler): Better extraction for amazon images | MohamedBassem | 2024-04-23 | 2 | -0/+3 | |
| | | ||||||
| * | fix(workers): Increase robustness of search worker and add extra logging. ↵ | MohamedBassem | 2024-04-23 | 1 | -24/+45 | |
| | | | | | Fixes #118 | |||||
| * | fix(workers): Set a modern user agent and update the default viewport size | MohamedBassem | 2024-04-23 | 1 | -0/+7 | |
| | | ||||||
| * | feature: Allow recrawling bookmarks without running inference jobs | MohamedBassem | 2024-04-20 | 1 | -7/+29 | |
| | | ||||||
| * | feature: Download images and screenshots | MohamedBassem | 2024-04-20 | 1 | -28/+130 | |
| | | ||||||
| * | fix: Fix slice call in the content truncation logic which was resulting in ↵ | MohamedBassem | 2024-04-15 | 1 | -1/+1 | |
| | | | | | excessive usage of context tokens. Fixes #94 | |||||
| * | feature: Add title to bookmarks and allow editing them. Fixes #27 | MohamedBassem | 2024-04-15 | 1 | -1/+2 | |
| | | ||||||
| * | fix: Differentiate between pending in db and in redis in admin job stats | MohamedBassem | 2024-04-12 | 1 | -1/+1 | |
| | | ||||||
| * | feature: Recrawl failed links from admin UI (#95) | Ahmad Mujahid | 2024-04-11 | 1 | -0/+20 | |
| | | | | | | * feature: Retry failed crawling URLs * fix: Enhancing visuals and some minor changes. | |||||
| * | fix: Increase default navigation timeout to 30s, make it configurable and ↵ | MohamedBassem | 2024-04-11 | 2 | -2/+1 | |
| | | | | | add retries to crawling jobs | |||||
| * | feature: Add PDF support (#88) | Ahmad Mujahid | 2024-04-11 | 4 | -12/+98 | |
| | | | | | | | | | | | | | | | | | | | | * feature: Add PDF support * fix: PDF feature enhancements * fix: Freeze expo-share-intent version to prevent breaking changes * fix: set endOfLine to auto for cross-platform development * fix: Upgrading eslint/parser and eslint-plugin to 7.6.0 to solve the linting issues * fix: enhancing PDF feature * fix: Allowing null in fiename for backward compatibility * fix: update pnpm file with pnpm 9.0.0-alpha-8 * fix:(web): PDF Preview for web | |||||
| * | feature(inference): Upgrade the default vision model to the new gpt-4-turbo | MohamedBassem | 2024-04-09 | 1 | -0/+1 | |
| | | ||||||
| * | fix(crawler): Skip validating URLs in metascrapper as it was already being ↵ | MohamedBassem | 2024-04-09 | 1 | -0/+3 | |
| | | | | | validated. Fixes #22 | |||||
| * | fix(workers): Increase default timeout to 60s, make it configurable and ↵ | MohamedBassem | 2024-04-06 | 1 | -11/+21 | |
| | | | | | improve logging | |||||
| * | feature: Include server version in the admin UI. Fixes #66 | MohamedBassem | 2024-04-02 | 1 | -0/+4 | |
| | | ||||||
| * | fix(workers): Add a timeout to the crawling job to prevent it from getting ↵ | MohamedBassem | 2024-04-02 | 2 | -1/+18 | |
| | | | | | stuck. Fixes #63 | |||||
| * | feat(workers): Allow configuring the language in which the tags are ↵ | MohamedBassem | 2024-04-02 | 1 | -5/+5 | |
| | | | | | generated. Fixes #68 | |||||
| * | chore(workers): Remove unused configuration options | MohamedBassem | 2024-03-31 | 1 | -2/+0 | |
| | | ||||||
| * | format: Add missing lint and format, and format the entire repo | MohamedBassem | 2024-03-30 | 7 | -25/+37 | |
| | | ||||||
| * | fix: Sort search results by relevance | MohamedBassem | 2024-03-30 | 1 | -0/+1 | |
| | | ||||||
| * | feature(web): Add support for attaching notes to bookmarks | MohamedBassem | 2024-03-30 | 1 | -0/+1 | |
| | | ||||||
| * | fix: Drop the 2k char limit on notes. Fixes #25 | MohamedBassem | 2024-03-27 | 1 | -6/+11 | |
| | | ||||||
| * | fix: Attempt to increase the reliability of the ollama inference | MohamedBassem | 2024-03-27 | 2 | -16/+40 | |
| | | ||||||
| * | feature: Add support for local models using ollama | MohamedBassem | 2024-03-27 | 4 | -76/+168 | |
| | | ||||||
| * | refactor: Validate env variables using zod | MohamedBassem | 2024-03-27 | 2 | -12/+12 | |
| | | ||||||
| * | docker: Use external chrome docker container | MohamedBassem | 2024-03-24 | 1 | -10/+40 | |
| | | ||||||
 |
index : karakeep | |
| Unnamed repository; edit this file 'description' to name the repository. |
| aboutsummaryrefslogtreecommitdiffstats |