What does the largest held document source hold, by issuing body, upload period and file type; how much of it carries a usable text layer; and what does a chunked, field-extracted pilot of it yield?

Portal office documents. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/objdocs.

Descriptive, sourced: no model

Basis: sourced records (URL, sha256 or locator per row), no model.

One row per stored file of the harvest manifest; the issuing body is the uploading portal office; links are exact-domain only; samples and pilot are measured, the remainder is counted, never estimated into served values except the labelled projection.

Stored files in the harvest: 71,769
Uploading portal offices: 4,213
Share of sampled PDFs with a text layer, 400 sampled: 40.25%
Pilot chunks extracted and field-read: 200

Which issuing bodies hold the most documents

The 25 offices with the most stored files

Portal	Files	First upload folder	Last upload folder
pressinform.gov.bd	Files: 2,197	First upload folder: 2024/12	Last upload folder: 2026/8
kwasa.gov.bd

How much carries a usable text layer

Share with a text layer, 400 PDFs sampled at random: 40.25%
PDFs text-checked so far: 1,893

The random sample by text-layer verdict

Verdict	PDFs
Text	PDFs: 161
No text

Are the stored bytes what was fetched

Files re-hashed at random: 400
Hashes that match: 400
Hashes that mismatch: 0
Files re-hashed so far: 538

How much links to twin objects

Offices matched to a registry body exactly: 490
Stored files under those offices: 1,691
Offices matched to a twin place exactly: 2,173
Stored files under those offices: 11,927

What the chunked pilot yields

Pilot documents: 103
Pilot pages: 480
Pilot chunks: 200
Numeric tokens in the chunks: 16,091
Pilot documents verified by re-hash: 103
Pilot documents printing a domain: 29
Pilot documents printing a kind: 14
Pilot documents printing a date

What the whole corpus would cost

cold runs only: the verify random sample (read and hash rate), the pilot-candidate text check, the pilot chunk run (single worker); text-layer PDFs estimated from the random text-check sample's text share

Whole-corpus projection: hours, each stage after the last: 2.87 hours
Whole-corpus projection: API cost: US$ 0
Read and hash every file: hours: 0.65 hours
Text-check every PDF: hours: 1.08 hours
Chunk the text-layer PDFs: hours, one worker: 1.14 hours
Text-layer PDFs estimated: 26,978

What remains

PDFs not text-checked: 65,134
Files not re-hashed: 71,231
Non-PDF files not parsed: 4,742
Offices without a pilot document: 4,110

Where these figures come from

The source collections

Source	Publisher	Unit	Caveat
Stored files from government portals	Publisher: Uploading offices of the national portal (stored copies)	Unit: one row per stored file: manifest line, URL, sha256 at fetch, bytes, office, upload folder, file type; source file is relative to the archive disk root	Caveat: upload folder is the object store's <year>/<number> folder exactly as it appears in the URL, not the document's own date; under 2026 the number runs 0 to 8, so it may be a zero-based month, which is not verified. The issuing body is the office-<slug> URL segment; its portal host is the slug with hyphens read as dots. Body and Place links are exact-domain matches only. sha256 verified is set only for re-hashed files (the random sample and the pilot); the rest carry the harvest's own sha256.

Not held

Document and chunk text: Never stored in the lake or served; chunks are located by page range and pinned by the sha256 of their extracted text so a reader re-extracts them.
Documents about a person: CVs, personal data sheets, rosters and beneficiary lists are screened out of the pilot; their printed dates and memo lines would describe individuals.
Phone numbers: Counted per chunk and document at the build (phones_detected), never stored.
A2 oracle_wave spec values: LLM-derived; used only to order pilot candidates, never served.

Type	Files	Bytes
pdf	Files: 67,027	Bytes: 110,343,570,344
doc	Files: 2,104	Bytes: 985,702,900
docx	Files: 1,786	Bytes: 346,114,776
xls	Files: 467	Bytes: 111,298,115
xlsx	Files: 249	Bytes: 53,558,004
pptx	Files: 122	Bytes: 247,859,551
ppt	Files: 12	Bytes: 21,989,888
csv	Files: 2	Bytes: 441,298

Upload folder	Files
2024/12	Files: 58,973
2025/11	Files: 10
2026/0	Files: 655
2026/1	Files: 607
2026/2	Files: 748
2026/3	Files: 1,296
2026/4	Files: 994
2026/5	Files: 1,309
2026/6	Files: 1,896
2026/7	Files: 2,195
2026/8	Files: 3,086

Kind	Documents
Tender	Documents: 4
Notification	Documents: 3
Policy	Documents: 2
Office order	Documents: 2
Budget	Documents: 1
Minutes	Documents: 1
Notice	Documents: 1

Check	Documents
None	Documents: 74
Forward host	Documents: 23
Other gov domain	Documents: 6

Pilot documents and their printed fields

Portal	Upload folder	Pages	Chunks	Printed domain	Printed kind	Printed date	Printed memo
jrcb.gov.bd	Upload folder: 2024/12	Pages: 3	Chunks: 3	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
hortex.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 1	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
brta.gov.bd	Upload folder: 2024/12	Pages: 6	Chunks: 1	Printed domain: www.brta.gov.bd	Printed kind: not available	Printed date: not available	Printed memo: ৩৫.০৩.০০০০.০০০.০০৮.৪০.০০১১.১৬.৩৪৪
iedcr.gov.bd	Upload folder: 2024/12	Pages: 6	Chunks: 3	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
fpmu.mofood.gov.bd	Upload folder: 2026/8	Pages: 4	Chunks: 3	Printed domain: mofood.gov.bd	Printed kind: not available	Printed date: ১৬.০৮.২০২৬	Printed memo: ১৩.০০.০০০০.০০০.০৬৭.৩৩.০০০১.২৪.৭৫
reb.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 3	Printed domain: not available	Printed kind: not available	Printed date: ১/৭/২০১৪	Printed memo: not available
asp.gov.bd	Upload folder: 2024/12	Pages: 23	Chunks: 2	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
bfsa.gov.bd	Upload folder: 2024/12	Pages: 39	Chunks: 2	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
comillaboard.gov.bd	Upload folder: 2026/7	Pages: 3	Chunks: 2	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
cwasa.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 4	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
rajshahieducationboard.gov.bd	Upload folder: 2024/12	Pages: 1	Chunks: 1	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
dmtcl.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 4	Printed domain: www.dmtcl.gov.bd	Printed kind: Tender	Printed date: not available	Printed memo: not available
heu.gov.bd	Upload folder: 2024/12	Pages: 5	Chunks: 4	Printed domain: not available	Printed kind: Budget	Printed date: not available	Printed memo: not available
mwb.gov.bd	Upload folder: 2026/8	Pages: 4	Chunks: 4	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
nrccb.gov.bd	Upload folder: 2024/12	Pages: 6	Chunks: 2	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
kurigramgc.college.gov.bd	Upload folder: 2024/12	Pages: 3	Chunks: 3	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
bba.gov.bd	Upload folder: 2026/8	Pages: 1	Chunks: 1	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
mohpw.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 2	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: not available
pressinform.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 2	Printed domain: not available	Printed kind: নীতিমালা	Printed date: ১৬.০৫.২০২৫	Printed memo: not available
molwa.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 1	Printed domain: not available	Printed kind: not available	Printed date: ৩০/০১/২০২১	Printed memo: not available
dot.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 1	Printed domain: www.dot.gov.bd	Printed kind: not available	Printed date: ০৬/০৪/২০২৫	Printed memo: ২৪.০২.০০০০.০০০.০০৩.১৪.০০০১.২৫.২৩
bicm.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 2	Printed domain: not available	Printed kind: Tender	Printed date: not available	Printed memo: not available
biman.airlines.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 1	Printed domain: not available	Printed kind: not available	Printed date: not available	Printed memo: ৩০.৩৪.০০০০.০৭৪.০২.০০৩.২১.৩৭৭
hil.gov.bd	Upload folder: 2024/12	Pages: 2	Chunks: 1	Printed domain: www.hil.gov.bd	Printed kind: not available	Printed date: ১-৩-২০২২	Printed memo: ৩০.৩৬.০০০০.০০৪.১৬.০০৮.১৮.৩৪
mpi.polytech.gov.bd	Upload folder: 2024/12	Pages: 4	Chunks: 4	Printed domain: not available	Printed kind: Minutes	Printed date: not available	Printed memo: not available

The first 25 of 103 rows the route serves.

Stage	Items	Seconds	Workers	Rate	Rate unit
Read	Items: 400	Seconds: 14.62	Workers: 4	Rate: 48,209,971.204	Rate unit: bytes read and hashed per second
Textcheck	Items: 1,493	Seconds: 86.267	Workers: 4	Rate: 17.307	Rate unit: PDFs text-checked per second
Chunk	Items: 469	Seconds: 71.662	Workers: 1	Rate: 6.545	Rate unit: PDFs extracted and chunked per second