fix: lazy load js-tiktoken in prompts module (#2176)

* feat: lazy load tiktoken to reduce memory footprint

The js-tiktoken module loads a large encoding dictionary into memory
immediately on import. This change defers the loading of the encoding
until it's actually needed by using a lazy getter pattern.

This reduces memory usage for processes that import this module but
don't actually use the token encoding functions.

* fix: use createRequire for lazy tiktoken import in ES module

The previous implementation used bare require() which fails at runtime
in ES modules (ReferenceError: require is not defined). This fixes it
by using createRequire from Node's 'module' package, which creates a
require function that works in ES module contexts.

* refactor: convert tiktoken lazy loading to async dynamic imports

Changed from createRequire to async import() for lazy loading tiktoken,
making buildTextPrompt and buildSummaryPrompt async. This is cleaner for
ES modules and properly defers the large tiktoken encoding data until
it's actually needed.

Updated all callers to await these async functions:
- packages/trpc/routers/bookmarks.ts
- apps/workers/workers/inference/tagging.ts
- apps/workers/workers/inference/summarize.ts
- apps/web/components/settings/AISettings.tsx (converted to useEffect)

* feat: add untruncated prompt builders for UI previews

Added buildTextPromptUntruncated and buildSummaryPromptUntruncated
functions that don't require token counting or truncation. These are
synchronous and don't load tiktoken, making them perfect for UI
previews where exact token limits aren't needed.

Updated AISettings.tsx to use these untruncated versions, eliminating
the need for useEffect/useState and avoiding unnecessary tiktoken
loading in the browser.

* fix

* fix

---------

Co-authored-by: Claude <noreply@anthropic.com>

1 files changed, 5 insertions, 6 deletions

diff --git a/apps/web/components/settings/AISettings.tsx b/apps/web/components/settings/AISettings.tsx
index 6ec71a91..beaa93dc 100644
--- a/apps/web/components/settings/AISettings.tsx
+++ b/apps/web/components/settings/AISettings.tsx
@@ -29,8 +29,8 @@ import { z } from "zod";
 
 import {
   buildImagePrompt,
-  buildSummaryPrompt,
-  buildTextPrompt,
+  buildSummaryPromptUntruncated,
+  buildTextPromptUntruncated,
 } from "@karakeep/shared/prompts";
 import {
   zNewPromptSchema,
@@ -301,6 +301,7 @@ export function PromptDemo() {
   const { t } = useTranslation();
   const { data: prompts } = api.prompts.list.useQuery();
   const clientConfig = useClientConfig();
+
   return (
     <div className="flex flex-col gap-2">
       <div className="mb-4 w-full text-xl font-medium sm:w-1/3">
@@ -308,7 +309,7 @@ export function PromptDemo() {
       </div>
       <p>{t("settings.ai.text_prompt")}</p>
       <code className="whitespace-pre-wrap rounded-md bg-muted p-3 text-sm text-muted-foreground">
-        {buildTextPrompt(
+        {buildTextPromptUntruncated(
           clientConfig.inference.inferredTagLang,
           (prompts ?? [])
             .filter(
@@ -316,7 +317,6 @@ export function PromptDemo() {
             )
             .map((p) => p.text),
           "\n<CONTENT_HERE>\n",
-          /* context length */ 1024 /* The value here doesn't matter */,
         ).trim()}
       </code>
       <p>{t("settings.ai.images_prompt")}</p>
@@ -332,13 +332,12 @@ export function PromptDemo() {
       </code>
       <p>{t("settings.ai.summarization_prompt")}</p>
       <code className="whitespace-pre-wrap rounded-md bg-muted p-3 text-sm text-muted-foreground">
-        {buildSummaryPrompt(
+        {buildSummaryPromptUntruncated(
           clientConfig.inference.inferredTagLang,
           (prompts ?? [])
             .filter((p) => p.appliesTo == "summary")
             .map((p) => p.text),
           "\n<CONTENT_HERE>\n",
-          /* context length */ 1024 /* The value here doesn't matter */,
         ).trim()}
       </code>
     </div>