Government publications by agency. Part of what the twin can answer. Every figure below is the value the route serves, with its source; the raw response is at /api/twin/brain/govfull/agency/bepza.
Descriptive, sourced: no model
Basis: sourced records (URL, sha256 or locator per row), no model.
stored files, documents and chunks, and counts over them
| Verdict | PDFs |
|---|---|
| Text | PDFs: 3 |
| Legacy bangla | PDFs: 1 |
| Manifest line | Type | Bytes | Characters | Also in the office store | Text layer | Pages | Re-hash verified |
|---|---|---|---|---|---|---|---|
| 239 | Type: pdf | Bytes: 1,655,825 |
| Pages | Chunks |
|---|---|
| 15 | Chunks: 14 |
Not held: Printed domain, Page, Line, Printed kind, Page, Line, Printed date, Page, Line, Printed memo, Page, Line.
| First page | Last page | Characters | Lines | Numeric tokens | Bangla share |
|---|---|---|---|---|---|
| 1 | Last page: 1 | Characters: 619 | Lines: 38 |
| Source | Publisher | Unit | Caveat |
|---|---|---|---|
| Government untruncated corpus harvest (stored copies, gap row A3) | Publisher: Issuing ministries, departments, divisions and boards of the Government of Bangladesh (stored copies) | Unit: one row per stored PDF document: manifest line, URL, sha256 at fetch, bytes, chars, issuing agency, portal host, upload year/month, registry entity binding, overlap with the stored files from government portals; source file is relative to the corpus root | Caveat: All 4,049 documents are full PDF publications; 241 match SHA-256 hashes of the stored files from government portals (BRN46). Agency assignment is verified against exact registry entities. sha256 verified is set only for re-hashed files (the 400 deterministic random sample and the pilot); the rest carry the harvest sha256. |
phones_detected), never stored.| Also in the office store: no |
| Text layer: Legacy bangla |
| Pages: 287 |
| Re-hash verified: yes |
| 306 | Type: pdf | Bytes: 25,285,712 | Characters: 59,811 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 459 | Type: pdf | Bytes: 7,190,562 | Characters: 319,964 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 467 | Type: pdf | Bytes: 15,524,051 | Characters: 181,507 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 796 | Type: pdf | Bytes: 3,440,775 | Characters: 84,988 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,288 | Type: pdf | Bytes: 21,683,186 | Characters: 63,998 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,348 | Type: pdf | Bytes: 3,454,182 | Characters: 86,920 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,395 | Type: pdf | Bytes: 19,716,152 | Characters: 72,105 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 1,870 | Type: pdf | Bytes: 3,420,208 | Characters: 74,586 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 2,519 | Type: pdf | Bytes: 2,335,084 | Characters: 1,064,350 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,163 | Type: pdf | Bytes: 24,820,817 | Characters: 192,579 | Also in the office store: no | Text layer: Text | Pages: 100 | Re-hash verified: yes |
| 3,316 | Type: pdf | Bytes: 14,397,779 | Characters: 201,115 | Also in the office store: no | Text layer: Text | Pages: 102 | Re-hash verified: yes |
| 3,348 | Type: pdf | Bytes: 711,456 | Characters: 177,063 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,393 | Type: pdf | Bytes: 3,418,309 | Characters: 66,835 | Also in the office store: no | Text layer: Text | Pages: 15 | Re-hash verified: yes |
| 3,560 | Type: pdf | Bytes: 595,152 | Characters: 201,441 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 3,793 | Type: pdf | Bytes: 74,311,251 | Characters: 194,015 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
| 4,028 | Type: pdf | Bytes: 2,625,658 | Characters: 66,517 | Also in the office store: no | Text layer: not available | Pages: not available | Re-hash verified: not available |
Not held: Upload year, Upload month.
| Bangla share: 0% |
| 2 | Last page: 2 | Characters: 4,622 | Lines: 106 | Numeric tokens: 10 | Bangla share: 0% |
| 3 | Last page: 3 | Characters: 1,977 | Lines: 53 | Numeric tokens: 9 | Bangla share: 0% |
| 4 | Last page: 4 | Characters: 3,567 | Lines: 84 | Numeric tokens: 16 | Bangla share: 0% |
| 5 | Last page: 5 | Characters: 3,004 | Lines: 59 | Numeric tokens: 18 | Bangla share: 0% |
| 6 | Last page: 6 | Characters: 2,155 | Lines: 56 | Numeric tokens: 16 | Bangla share: 0% |
| 7 | Last page: 7 | Characters: 3,859 | Lines: 116 | Numeric tokens: 39 | Bangla share: 0% |
| 8 | Last page: 8 | Characters: 4,685 | Lines: 96 | Numeric tokens: 12 | Bangla share: 0% |
| 9 | Last page: 9 | Characters: 4,294 | Lines: 77 | Numeric tokens: 31 | Bangla share: 0% |
| 10 | Last page: 10 | Characters: 3,408 | Lines: 72 | Numeric tokens: 26 | Bangla share: 0% |
| 11 | Last page: 11 | Characters: 2,619 | Lines: 59 | Numeric tokens: 25 | Bangla share: 0% |
| 12 | Last page: 12 | Characters: 2,584 | Lines: 53 | Numeric tokens: 16 | Bangla share: 0% |
| 13 | Last page: 14 | Characters: 3,521 | Lines: 250 | Numeric tokens: 178 | Bangla share: 0% |
| 15 | Last page: 15 | Characters: 505 | Lines: 24 | Numeric tokens: 22 | Bangla share: 0% |
| Per-issuing-agency manifest datasets | Publisher: Built from the rows above by the inventory pipeline | Unit: one row per issuing agency, with its partition file, sha256, file counts, and text layer counts | Caveat: Each agency's inventory partition is the resumable unit of the pipeline; a rerun rewrites only partitions whose rows changed. |
| 200-chunk pilot: documents and chunks with printed fields across 14 agencies | Publisher: each document's stored PDF (PyMuPDF text layer) | Unit: one row per pilot document (fields with page and line) and per chunk (page range, counts, text sha256) | Caveat: Pilot documents were chosen to maximize agency diversity across text-layer candidates, chunked in consecutive page blocks up to 4,000 characters, hitting exactly 200 chunks total. Documents with personal data (CVs, data sheets, rosters, beneficiary lists, or >=2 mobile phone numbers on pages 1-2) are screened out before chunking. Phone numbers are never stored in any parquet or served payload. |
| Pipeline run log (measured) | Publisher: Measured pipeline runs on a 16-core machine | Unit: one row per stage run: items, bytes, seconds, workers | Caveat: Runs over files a previous run had read hit the page cache and overstate the rate; the projection uses the cold runs named in its basis. |