Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/dshe.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 33 | Bytes: 70,759,770 |
| 2026 | Files: 8 | Bytes: 23,082,430 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 5 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 71 | Type: pdf |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 36,020 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: yes |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 118 | Type: pdf | Bytes: 594,108 | Characters: 42,993 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 167 | Type: pdf | Bytes: 897,750 | Characters: 48,611 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 247 | Type: pdf | Bytes: 4,230,455 | Characters: 405,514 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 252 | Type: pdf | Bytes: 295,732 | Characters: 43,955 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 395 | Type: pdf | Bytes: 2,843,617 | Characters: 671,285 | Upload year: 2026 | Upload month: 7 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 553 | Type: pdf | Bytes: 2,063,292 | Characters: 593,702 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 556 | Type: pdf | Bytes: 2,838,951 | Characters: 268,202 | Upload year: 2026 | Upload month: 4 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 586 | Type: pdf | Bytes: 2,505,742 | Characters: 232,071 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: Text | Pages: 78 | Re-hash verified: yes |
| 632 | Type: pdf | Bytes: 2,138,760 | Characters: 158,640 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 650 | Type: pdf | Bytes: 1,253,408 | Characters: 220,966 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 664 | Type: pdf | Bytes: 3,014,782 | Characters: 246,391 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 677 | Type: pdf | Bytes: 2,212,873 | Characters: 580,125 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 741 | Type: pdf | Bytes: 2,073,279 | Characters: 574,031 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 122 | Re-hash verified: yes |
| 749 | Type: pdf | Bytes: 289,637 | Characters: 40,502 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 836 | Type: pdf | Bytes: 1,886,871 | Characters: 76,385 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 865 | Type: pdf | Bytes: 1,335,867 | Characters: 52,602 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 896 | Type: pdf | Bytes: 5,409,457 | Characters: 2,592,999 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: Text | Pages: 743 | Re-hash verified: yes |
| 937 | Type: pdf | Bytes: 290,989 | Characters: 56,608 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 979 | Type: pdf | Bytes: 2,135,828 | Characters: 581,060 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 96 | Re-hash verified: yes |
| 1,363 | Type: pdf | Bytes: 2,843,617 | Characters: 671,285 | Upload year: 2026 | Upload month: 7 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,426 | Type: pdf | Bytes: 2,629,940 | Characters: 49,465 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,451 | Type: pdf | Bytes: 645,219 | Characters: 37,043 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,684 | Type: pdf | Bytes: 291,950 | Characters: 45,537 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,889 | Type: pdf | Bytes: 3,186,929 | Characters: 653,844 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,996 | Type: pdf | Bytes: 514,782 | Characters: 29,201 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,021 | Type: pdf | Bytes: 1,568,271 | Characters: 242,315 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,275 | Type: pdf | Bytes: 1,253,408 | Characters: 220,966 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,657 | Type: pdf | Bytes: 2,505,171 | Characters: 198,670 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,757 | Type: pdf | Bytes: 11,789,332 | Characters: 129,558 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,923 | Type: pdf | Bytes: 180,308 | Characters: 40,191 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,179 | Type: pdf | Bytes: 2,478,580 | Characters: 236,108 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,301 | Type: pdf | Bytes: 1,626,894 | Characters: 54,869 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,488 | Type: pdf | Bytes: 2,138,760 | Characters: 158,640 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,489 | Type: pdf | Bytes: 3,803,284 | Characters: 576,739 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,532 | Type: pdf | Bytes: 2,505,171 | Characters: 198,670 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,561 | Type: pdf | Bytes: 2,088,588 | Characters: 630,343 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,602 | Type: pdf | Bytes: 2,140,318 | Characters: 574,346 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,648 | Type: pdf | Bytes: 4,230,455 | Characters: 405,514 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,687 | Type: pdf | Bytes: 1,213,313 | Characters: 75,468 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: Text | Pages: 34 | Re-hash verified: yes |
| 3,807 | Type: pdf | Bytes: 1,624,449 | Characters: 62,293 | Upload year: 2026 | Upload month: 4 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |