Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/mincom.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 38 | Bytes: 52,723,208 |
| 2026 | Files: 8 | Bytes: 9,667,087 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 7 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 114 | Type: pdf |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 414,781 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 245 | Type: pdf | Bytes: 222,393 | Characters: 395,862 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 324 | Type: pdf | Bytes: 3,175,813 | Characters: 527,885 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 389 | Type: pdf | Bytes: 236,527 | Characters: 415,751 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 409 | Type: pdf | Bytes: 700,189 | Characters: 181,183 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 433 | Type: pdf | Bytes: 1,198,635 | Characters: 254,494 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 685 | Type: pdf | Bytes: 193,857 | Characters: 312,811 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 693 | Type: pdf | Bytes: 689,557 | Characters: 80,106 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 712 | Type: pdf | Bytes: 2,275,449 | Characters: 184,366 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,053 | Type: pdf | Bytes: 228,111 | Characters: 417,930 | Upload year: 2026 | Upload month: 1 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,435 | Type: pdf | Bytes: 243,119 | Characters: 436,285 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,531 | Type: pdf | Bytes: 703,148 | Characters: 44,039 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,549 | Type: pdf | Bytes: 295,797 | Characters: 103,879 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,669 | Type: pdf | Bytes: 1,268,068 | Characters: 132,253 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,115 | Type: pdf | Bytes: 238,092 | Characters: 421,796 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,181 | Type: pdf | Bytes: 73,794 | Characters: 58,291 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,185 | Type: pdf | Bytes: 1,106,851 | Characters: 235,274 | Upload year: 2026 | Upload month: 3 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,297 | Type: pdf | Bytes: 2,005,899 | Characters: 259,667 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,409 | Type: pdf | Bytes: 190,081 | Characters: 301,018 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,499 | Type: pdf | Bytes: 328,002 | Characters: 70,339 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 21 | Re-hash verified: yes |
| 2,550 | Type: pdf | Bytes: 341,805 | Characters: 119,688 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,629 | Type: pdf | Bytes: 498,361 | Characters: 75,306 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,740 | Type: pdf | Bytes: 1,151,054 | Characters: 234,723 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,802 | Type: pdf | Bytes: 328,449 | Characters: 88,729 | Upload year: 2026 | Upload month: 3 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,912 | Type: pdf | Bytes: 723,681 | Characters: 207,713 | Upload year: 2026 | Upload month: 7 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,948 | Type: pdf | Bytes: 1,723,855 | Characters: 93,415 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,069 | Type: pdf | Bytes: 2,135,433 | Characters: 162,839 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 43 | Re-hash verified: yes |
| 3,070 | Type: pdf | Bytes: 861,542 | Characters: 147,844 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 52 | Re-hash verified: yes |
| 3,108 | Type: pdf | Bytes: 1,151,858 | Characters: 239,057 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 46 | Re-hash verified: yes |
| 3,113 | Type: pdf | Bytes: 446,423 | Characters: 93,178 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,199 | Type: pdf | Bytes: 424,114 | Characters: 93,156 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,210 | Type: pdf | Bytes: 2,195,804 | Characters: 178,660 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,222 | Type: pdf | Bytes: 346,709 | Characters: 83,821 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,356 | Type: pdf | Bytes: 2,054,158 | Characters: 536,175 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,526 | Type: pdf | Bytes: 691,595 | Characters: 69,755 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,549 | Type: pdf | Bytes: 1,100,452 | Characters: 231,591 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,570 | Type: pdf | Bytes: 232,438 | Characters: 406,725 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,585 | Type: pdf | Bytes: 325,262 | Characters: 69,850 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,624 | Type: pdf | Bytes: 1,129,924 | Characters: 238,910 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,633 | Type: pdf | Bytes: 944,151 | Characters: 257,021 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 29 | Re-hash verified: yes |
| 3,717 | Type: pdf | Bytes: 6,242,718 | Characters: 4,381,662 | Upload year: 2026 | Upload month: 1 | Also in the office store: no | Text layer: Text | Pages: 1,272 | Re-hash verified: yes |
| 3,775 | Type: pdf | Bytes: 184,961 | Characters: 304,389 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 55 | Re-hash verified: yes |
| 3,826 | Type: pdf | Bytes: 687,362 | Characters: 106,012 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,879 | Type: pdf | Bytes: 373,719 | Characters: 65,205 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,907 | Type: pdf | Bytes: 15,700,444 | Characters: 1,400,732 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,934 | Type: pdf | Bytes: 4,782,103 | Characters: 355,970 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |