Remote http(s) images written as <img src="..."> reached no resolver.
Markdown images went through ResolveRemoteImages, HTML data-URIs and HTML
relative paths each had their own pass, but nothing claimed the remaining
combination, so those images were dropped without a log line and the
multimodal stage was skipped for documents that rely on HTML for layout.
ResolveRemoteImages now scans HTML img tags with a quoted src as a second
pass and rewrites only the src attribute value, leaving the tag intact
because the surrounding markup is why these documents use HTML at all.
The per-image work (whitelist, SSRF, download, icon filter, upload) moves
into one helper so the two syntaxes cannot drift apart on policy. Each
pass carries its own image budget; sharing one would let HTML images eat
into the number of Markdown images a document already gets resolved.
An attribute value is trimmed, HTML-unescaped and its scheme lowercased
before use, because HTML permits padding, spells a query separator &
and tolerates any scheme case, while every fetcher downstream compares the
scheme byte-for-byte. Markdown targets get none of that normalization, so
no Markdown reference resolves differently than it does today. A
whitelisted host keeps serving the image from where it already lives, and
its span is rewritten with the normalized URL rather than the raw attribute
text: later stages both locate the image by searching the document for its
serving URL and fetch that same string, so the two have to agree.
The src attribute must now be preceded by whitespace, so a hyphenated
attribute name is no longer mistaken for it. Among HTML tags this cuts two
ways: one carrying both a hyphenated *-src and a real src now resolves the
real src instead of the placeholder, while one carrying only a hyphenated
*-src is no longer resolved at all. Recovering the latter needs an
attribute parser rather than a regex, and resolving a lazy-loading
placeholder while the real image goes unvisited is the worse of the two
failures. Both halves are pinned by tests.
Image-derived text also has to survive into the answer, which is why this
touches internal/searchutil. The chat enricher and the image_info filters
matched Markdown links only, so an HTML-only passage looked image-free:
every entry was dropped and the model received bare markup with the
caption and OCR text stripped, even though the analysis had run. Both now
recognise HTML img tags through one shared regex, so a tag the parser
stores can always be matched back. Both syntaxes are located against the
original passage and spliced in a single pass; scanning one regex over the
other's output let a metadata block that itself quotes an image reference
be rescanned and duplicated. The chat variant still enriches only images
present in the passage, so orphan entries stay out.
Known limits:
- An unquoted src and a srcset-only tag are not matched, nor is a tag
whose attribute values hold a bare > or a quoted src= of their own.
- imgHTMLDataURI still matches src without a boundary, so a data-src
holding a base64 placeholder is consumed as if it were the src and the
whole tag replaced. Worth the same fix, but it changes what an existing
lazy-loading document resolves, so it belongs on its own.
- A reference shown as a documentation example is rewritten like any
other. Recognising code spans needs a Markdown parser; a hand-rolled
scanner risks the far worse failure of treating a document's real
screenshots as examples.
- Rewriting a whitelisted src writes the unescaped URL back into an HTML
attribute, so a query parameter whose name spells an entity renders
differently than the author wrote it. Keeping the encoded form would
mean unescaping on every matcher instead.
- Image text reaches the model only when the whole reference falls inside
the retrieved window; an img tag is several times longer than a
Markdown link and straddles a chunk boundary more often.
- A reference repeated in one document is fetched once per occurrence,
which is what gives each chunk its own analysed copy.
- Nothing bounds how long one document's fetches may take in total.
- Parent-context pruning removes only Markdown images.
Co-authored-by: wizardchen <wizardchen@tencent.com>
Three follow-up fixes on top of the MinerU markdown preservation work:
- Stop applying normalizeMinerUMarkdown inside ResolveAndStore. The
helper is already called by MinerUReader.Read, and ResolveAndStore is
shared by every parser (docreader, session attachments, ...). Running
the heading/image unescape regexes globally would silently rewrite
content (including inside fenced code blocks) for non-MinerU sources.
- Recognize MinerU image references whose path contains spaces, e.g.
"images/第 1 页.jpg". The previous regex used in
extractImageRefsFromContent disallowed whitespace in the URL group,
so such images were never matched and never persisted. Use a
whitespace-tolerant pattern aligned with ResolveAndStore's own
imgPattern.
- Deduplicate uploads when the same MinerU image is referenced under
multiple path forms (e.g. "images/foo.png" vs "./images/foo.png").
saveReferencedImage now caches by ref.Filename in addition to the
raw ref path, so the second variant reuses the previously stored
ServingURL instead of writing the same bytes to object storage
again.
Tests added:
- TestProcessImagesMatchesPathsWithSpaces
- TestResolveAndStoreDedupsSameImageRefVariants
MinerU already returns markdown with embedded HTML blocks, but the current\nreader runs the whole document back through html-to-markdown. That\nsecond conversion escapes valid headings and image syntax, so chunk\nprofiling sees plain text instead of markdown structure and relative\nimage references stop matching the storage pipeline.\n\nKeep MinerU output in its original markdown form and only apply narrow\ncompatibility normalization for the specific over-escaped patterns we\nactually need to recover. The converter now matches image refs by the\npaths that are really present in markdown or embedded HTML instead of\nassuming a single images/<name> form.\n\nExtend ImageResolver so relative HTML <img src=...> references share the\nsame storage rewrite path as markdown images, deduplicate repeated saves,\nand keep the frontend sanitizer compatible with MinerU's details/summary\nblocks. Add focused docparser tests that cover escaped markdown repair,\nvariant image path matching, and relative HTML image persistence.
When the uploaded file is itself an image, the image reference now carries
an IsOriginal flag so ResolveAndStore skips the small-icon size filter.
Otherwise small standalone images (e.g. avatars below 64x64) were silently
dropped before reaching multimodal OCR/caption processing.
- Enhanced the logging of the tenant access token in the Feishu client to handle variable lengths for prefix and suffix, ensuring accurate display even for shorter tokens.
- Updated the test cases in `connector_test.go` to reflect changes in the endpoint path for file downloads, improving test reliability.
- Adjusted example outputs in `example_test.go` for clarity and consistency.
- Modified image resolution tests to correct expected outcomes based on updated criteria for icon images.
- Added environment variable settings in image resolver tests to ensure localhost accessibility during testing.
These changes enhance the robustness of logging and testing mechanisms across the application.
- Keep strict mode blocking all direct IPs (IPv4 and IPv6 uniformly)
- Unify all SSRF call sites to use ValidateURLForSSRF (whitelist-aware)
- Add Teredo (2001:0000::/32) and 6to4 (2002::/16) tunnel detection
- Make redirect handler and DNS pinning respect SSRF_WHITELIST
- Unexport isSSRFSafeURL to prevent future callers bypassing whitelist
- Add scheme validation for whitelisted redirect targets
- Document IPv6 whitelist syntax in .env.example
- Add comprehensive IPv6 test coverage
- Introduced a new task enqueuer in the ImageMultimodalService to handle question generation for image knowledge.
- Implemented logic to enqueue question generation tasks when valid captions or OCR content are available, enhancing the knowledge base's interactivity.
- Updated image reference handling to ensure markdown URLs are correctly formatted, accommodating filenames with spaces.
- Enhanced regex pattern in image resolver to support filenames with spaces, improving image processing accuracy.
- Updated regex patterns in MarkdownImageUtil to support alt text containing brackets and handle MIME types with hyphens.
- Implemented new functions in ImageResolver for resolving HTML <img> tags with data URIs and bare base64 content, improving image handling in markdown.
- Added comprehensive tests for various image scenarios, ensuring robust handling of data URIs and base64 images.
- Added functionality to resolve embedded data:image;base64 images in markdown, storing them and replacing references with stable URLs.
- Updated the knowledge service to process data URIs before handling remote images, ensuring all images are resolved consistently.
- Improved logging for resolved images, providing better insights during manual processing.
- Added unit tests to validate the new data URI resolution functionality and ensure reliability.
- Introduced a new function `isIconImage` to identify small images and decorative elements based on pixel dimensions and file size.
- Updated the image resolution logic to filter out identified icon images from markdown content, enhancing the quality of stored images.
- Added unit tests for icon image detection to ensure accurate functionality across various image sizes.
These changes improve image handling by effectively filtering out non-essential images, leading to cleaner document processing.
- Removed the image storage handling from the `_resolve_images` function, which now only decodes images and returns them as inline bytes.
- Updated the function's docstring to clarify that image persistence is managed by the Go App, and the return value for `image_dir_path` is always empty.
- Adjusted related gRPC and protobuf files to remove the `ConvertToPDF` method and its associated types, streamlining the API.
- Enhanced security measures in the frontend by implementing a placeholder for provider images and ensuring proper hydration of protected file images.
These changes improve the clarity and efficiency of image processing and API interactions within the application.
- Introduced a loading state (`loadingChunks`) to manage document chunk loading more effectively.
- Updated the scroll handling logic to prevent multiple requests while loading chunks.
- Adjusted the pagination logic to ensure correct page increments based on the total document count.
This update improves the user experience by ensuring smoother document loading and better state management during scrolling interactions.
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.
This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.