Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/dpe.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 36 | Bytes: 70,434,650 |
| 2026 | Files: 10 | Bytes: 20,259,629 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 4 |
| Legacy bangla | PDFs: 2 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 22 | Type: pdf |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 59,153 |
| Upload year: 2026 |
| Upload month: 6 |
| Also in the office store: no |
| Text layer: Text |
| Pages: 34 |
| Re-hash verified: yes |
| 221 | Type: pdf | Bytes: 793,738 | Characters: 85,476 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 242 | Type: pdf | Bytes: 135,911 | Characters: 83,965 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 259 | Type: pdf | Bytes: 12,907,436 | Characters: 297,920 | Upload year: 2026 | Upload month: 0 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 350 | Type: pdf | Bytes: 1,127,810 | Characters: 184,410 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 445 | Type: pdf | Bytes: 341,003 | Characters: 70,525 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 454 | Type: pdf | Bytes: 1,287,812 | Characters: 160,961 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 626 | Type: pdf | Bytes: 3,212,976 | Characters: 175,136 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 816 | Type: pdf | Bytes: 1,139,663 | Characters: 90,886 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 866 | Type: pdf | Bytes: 2,980,460 | Characters: 228,252 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 114 | Re-hash verified: yes |
| 925 | Type: pdf | Bytes: 1,381,121 | Characters: 101,714 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 962 | Type: pdf | Bytes: 172,344 | Characters: 30,107 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,076 | Type: pdf | Bytes: 1,515,988 | Characters: 240,444 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,120 | Type: pdf | Bytes: 865,005 | Characters: 87,141 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,191 | Type: pdf | Bytes: 3,957,932 | Characters: 641,893 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,193 | Type: pdf | Bytes: 1,568,271 | Characters: 242,315 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,339 | Type: pdf | Bytes: 175,164 | Characters: 32,745 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,498 | Type: pdf | Bytes: 118,591 | Characters: 71,602 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,582 | Type: pdf | Bytes: 846,794 | Characters: 84,221 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,671 | Type: pdf | Bytes: 2,394,467 | Characters: 331,292 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,741 | Type: pdf | Bytes: 310,136 | Characters: 136,637 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,879 | Type: pdf | Bytes: 2,610,068 | Characters: 566,896 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,227 | Type: pdf | Bytes: 939,421 | Characters: 98,513 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,410 | Type: pdf | Bytes: 174,026 | Characters: 30,335 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,557 | Type: pdf | Bytes: 1,483,444 | Characters: 276,583 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,759 | Type: pdf | Bytes: 852,745 | Characters: 248,075 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,902 | Type: pdf | Bytes: 1,552,421 | Characters: 85,128 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Legacy bangla | Pages: 44 | Re-hash verified: yes |
| 2,985 | Type: pdf | Bytes: 204,332 | Characters: 50,686 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,046 | Type: pdf | Bytes: 109,433 | Characters: 63,627 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,084 | Type: pdf | Bytes: 2,228,721 | Characters: 357,998 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,088 | Type: pdf | Bytes: 2,602,045 | Characters: 78,707 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,107 | Type: pdf | Bytes: 329,616 | Characters: 67,435 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,173 | Type: pdf | Bytes: 6,513,707 | Characters: 606,377 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,194 | Type: pdf | Bytes: 5,068,293 | Characters: 747,242 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,303 | Type: pdf | Bytes: 3,001,429 | Characters: 6,490,640 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,418 | Type: pdf | Bytes: 1,684,780 | Characters: 1,035,121 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,472 | Type: pdf | Bytes: 1,982,030 | Characters: 122,910 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Legacy bangla | Pages: 64 | Re-hash verified: yes |
| 3,509 | Type: pdf | Bytes: 181,097 | Characters: 35,500 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,591 | Type: pdf | Bytes: 2,461,447 | Characters: 199,943 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,628 | Type: pdf | Bytes: 330,817 | Characters: 85,286 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,698 | Type: pdf | Bytes: 217,564 | Characters: 66,715 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: Text | Pages: 30 | Re-hash verified: yes |
| 3,721 | Type: pdf | Bytes: 2,432,657 | Characters: 536,724 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,812 | Type: pdf | Bytes: 9,851,829 | Characters: 775,768 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,915 | Type: pdf | Bytes: 1,484,322 | Characters: 103,125 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,963 | Type: pdf | Bytes: 3,009,248 | Characters: 6,477,562 | Upload year: 2026 | Upload month: 5 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 4,030 | Type: pdf | Bytes: 1,939,176 | Characters: 343,499 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 124 | Re-hash verified: yes |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |