How to read the published columns unchanged

Call get_tossd_raw to get a year of TOSSD data with the publisher's original column names, dtypes, and ordering across every row.

Use cases for raw published data

get_tossd_raw serves three primary use cases:

Inspecting raw published records directly before relying on normalised columns.
Isolating specific values when diagnosing a SchemaDriftError, which triggers during normalisation.
Reconciling totals against the International Forum on TOSSD (IFT) portal at tossd.online, which displays raw published headers and amounts.

Steps

Query data with get_tossd_raw. The function accepts years and refresh parameters.
```
import tossd_reader as tossd

raw = tossd.get_tossd_raw(years=2024)
raw.shape
```
```
(474026, 53)
```
The resulting frame contains 474,026 rows, matching the full row count of get_tossd(years=2024), across all 53 original published columns.

Inspect the column names. The frame retains the publisher's unrenamed headers.

list(raw.columns[:10])

['Year', 'provider', 'ProviderNameE', 'agencyname_E', 'tossdid', 'ProjectNumber', 'recipientcode', 'recipientnamee', 'regionnamee', 'Channel']

The first three columns correspond to year, provider_code, and provider_name in get_tossd.

Check the data types. Every column is stored as str or float64, representing the direct output of parquet reading without schema casting.
```
raw.dtypes.value_counts()
```
```
str        44
float64     9
Name: count, dtype: int64
```
A code column delivered as a nullable integer in get_tossd remains a string in the raw frame.
```
raw["provider"].dtype
```
```
<StringDtype(na_value=nan)>
```
```
raw.loc[5, "provider"]
```
```
'4'
```
Compare results with get_tossd. get_tossd returns the same rows with typed columns and a focused set under the "minimal" preset.
```
h = tossd.get_tossd(years=2024, columns="minimal")
h.shape
```
```
(474026, 19)
```
```
h["provider_code"].dtype
```
```
Int16Dtype()
```
```
int(h.loc[5, "provider_code"])
```
```
4
```
raw.loc[5, "provider"] and h.loc[5, "provider_code"] hold the same provider code (4), represented respectively as a string and an integer.

Verify it worked

get_tossd_raw delivers unfiltered rows in the publisher's original units.

Without provider filtering, the aggregate pseudo-provider rows (code 0) remain in the frame, matching the count in get_tossd for the same year.

int((raw["provider"] == "0").sum())

Amounts remain in the publisher's original scale, matching get_tossd's default of units="usd_thousand".

float(raw.loc[5, "USD_disbursements"])

10.815487778498811

float(h.loc[5, "usd_disbursement"])

10.815487778498811

get_tossd_raw preserves empty strings from the source file, whereas get_tossd converts them to nulls.

int((raw["ProjectNumber"] == "").sum())

int(h["project_number"].isna().sum())

Absence of derived columns and filters in raw frames

A frame from get_tossd_raw provides only the raw published columns. Filter and convert units in pandas, or query get_tossd to get typed columns, the derived is_aggregate flag, and built-in filters.

Troubleshooting

TypeError naming an unexpected keyword argument. Passing filter arguments such as providers= or recipients= to get_tossd_raw raises a TypeError.

tossd.get_tossd_raw(years=2024, providers="Senegal")

TypeError: get_tossd_raw() got unexpected keyword argument(s): providers. get_tossd_raw() only accepts years=/refresh=; for filtering, column selection, or unit conversion, use get_tossd() instead.

How to read the published columns unchanged

Steps

Verify it worked

Troubleshooting

See also