Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/epb.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Upload year | Files | Bytes |
|---|---|---|
| 2024 | Files: 62 | Bytes: 99,179,766 |
| 2026 | Files: 5 | Bytes: 30,600,251 |
| Verdict | PDFs |
|---|---|
| Text | PDFs: 5 |
| Undecodable | PDFs: 1 |
| Manifest line | Type | Bytes | Characters | Upload year | Upload month | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|---|---|
| 35 | Type: pdf |
| Pages | Chunks |
|---|---|
| 17 | Chunks: 6 |
Not held: Printed domain, Page, Line, Printed kind, Page, Line, Printed date, Page, Line, Printed memo, Page, Line.
| First page | Last page | Characters | Lines | Numeric tokens | Bangla share |
|---|---|---|---|---|---|
| 1 | Last page: 2 | Characters: 1,602 | Lines: 40 |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Characters: 195,111 |
| Upload year: 2024 |
| Upload month: 12 |
| Also in the office store: no |
| Text layer: not available |
| Pages: not available |
| Re-hash verified: not available |
| 87 | Type: pdf | Bytes: 232,796 | Characters: 52,512 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 244 | Type: pdf | Bytes: 3,964,989 | Characters: 324,854 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 431 | Type: pdf | Bytes: 3,164,971 | Characters: 691,094 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 449 | Type: pdf | Bytes: 1,661,305 | Characters: 195,111 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 533 | Type: pdf | Bytes: 380,388 | Characters: 61,198 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 584 | Type: pdf | Bytes: 283,349 | Characters: 38,165 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 17 | Re-hash verified: yes |
| 763 | Type: pdf | Bytes: 367,689 | Characters: 254,668 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 72 | Re-hash verified: yes |
| 779 | Type: pdf | Bytes: 380,388 | Characters: 61,198 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 927 | Type: pdf | Bytes: 1,177,433 | Characters: 291,024 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 973 | Type: pdf | Bytes: 579,631 | Characters: 77,387 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,038 | Type: pdf | Bytes: 241,150 | Characters: 103,300 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,126 | Type: pdf | Bytes: 361,835 | Characters: 40,348 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,141 | Type: pdf | Bytes: 313,362 | Characters: 56,920 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,152 | Type: pdf | Bytes: 767,457 | Characters: 70,838 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 25 | Re-hash verified: yes |
| 1,190 | Type: pdf | Bytes: 720,372 | Characters: 152,502 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: Undecodable | Pages: 72 | Re-hash verified: yes |
| 1,214 | Type: pdf | Bytes: 232,796 | Characters: 52,512 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,228 | Type: pdf | Bytes: 4,793,404 | Characters: 144,137 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,236 | Type: pdf | Bytes: 3,769,543 | Characters: 1,133,702 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,242 | Type: pdf | Bytes: 756,066 | Characters: 60,982 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,270 | Type: pdf | Bytes: 368,076 | Characters: 82,776 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,329 | Type: pdf | Bytes: 784,261 | Characters: 77,252 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,495 | Type: pdf | Bytes: 6,264,862 | Characters: 112,577 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,527 | Type: pdf | Bytes: 756,066 | Characters: 60,982 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,541 | Type: pdf | Bytes: 4,160,299 | Characters: 1,869,144 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,579 | Type: pdf | Bytes: 8,740,530 | Characters: 2,764,436 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,647 | Type: pdf | Bytes: 3,951,544 | Characters: 95,224 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,736 | Type: pdf | Bytes: 1,127,043 | Characters: 497,538 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,764 | Type: pdf | Bytes: 302,506 | Characters: 53,373 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,786 | Type: pdf | Bytes: 536,404 | Characters: 57,831 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 12 | Re-hash verified: yes |
| 1,882 | Type: pdf | Bytes: 182,052 | Characters: 86,526 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,123 | Type: pdf | Bytes: 510,216 | Characters: 119,928 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,136 | Type: pdf | Bytes: 323,810 | Characters: 65,985 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,150 | Type: pdf | Bytes: 883,815 | Characters: 68,451 | Upload year: 2026 | Upload month: 4 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,236 | Type: pdf | Bytes: 504,181 | Characters: 108,177 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,590 | Type: pdf | Bytes: 753,748 | Characters: 328,902 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,646 | Type: pdf | Bytes: 2,341,412 | Characters: 38,208 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,650 | Type: pdf | Bytes: 11,737,015 | Characters: 283,924 | Upload year: 2026 | Upload month: 2 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,695 | Type: pdf | Bytes: 3,964,989 | Characters: 324,854 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,738 | Type: pdf | Bytes: 355,213 | Characters: 38,263 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,744 | Type: pdf | Bytes: 2,705,677 | Characters: 173,577 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,749 | Type: pdf | Bytes: 700,189 | Characters: 181,183 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,791 | Type: pdf | Bytes: 368,076 | Characters: 82,776 | Upload year: 2024 | Upload month: 12 | Also in the office store: yes | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,883 | Type: pdf | Bytes: 288,097 | Characters: 43,034 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,928 | Type: pdf | Bytes: 498,361 | Characters: 75,306 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,932 | Type: pdf | Bytes: 301,319 | Characters: 53,053 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,955 | Type: pdf | Bytes: 8,740,530 | Characters: 2,764,436 | Upload year: 2026 | Upload month: 6 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,032 | Type: pdf | Bytes: 505,351 | Characters: 49,181 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,047 | Type: pdf | Bytes: 4,700,857 | Characters: 91,807 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: Text | Pages: 52 | Re-hash verified: yes |
| 3,132 | Type: pdf | Bytes: 355,544 | Characters: 38,424 | Upload year: 2024 | Upload month: 12 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| Bangla share: 87.99% |
| 3 | Last page: 4 | Characters: 3,059 | Lines: 48 | Numeric tokens: 45 | Bangla share: 79.76% |
| 5 | Last page: 8 | Characters: 3,913 | Lines: 253 | Numeric tokens: 167 | Bangla share: 89.4% |
| 9 | Last page: 11 | Characters: 3,790 | Lines: 365 | Numeric tokens: 215 | Bangla share: 93.79% |
| 12 | Last page: 14 | Characters: 3,655 | Lines: 147 | Numeric tokens: 83 | Bangla share: 76.06% |
| 15 | Last page: 17 | Characters: 1,574 | Lines: 60 | Numeric tokens: 34 | Bangla share: 90.46% |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |