Benchmarking Results
Performance metrics on 10,000,000 rows. Speedup factors demonstrate the efficiency of kdb+ offloading for large-scale operations.
Core
DataFrame Creation
qutePandas: 1.8157 s
Speedup: 5.9x
Pandas: 0.0009s | qutePandas: 0.0002s
Indexing
loc (boolean mask)
Speedup: 11.0x
Pandas: 0.54s | qutePandas: 0.0494s
iloc (rows slice)
Speedup: 0.1311x
Pandas: 0.0001s | qutePandas: 0.0006s
iloc (cols slice)
Speedup: 938.1x
Pandas: 0.13s | qutePandas: 0.0001s
Cleaning
dropna
Speedup: 8.6x
Pandas: 1.14s | qutePandas: 0.1322s
dropna_col
Speedup: 12.4x
Pandas: 0.62s | qutePandas: 0.0503s
fillna
Speedup: 178.6x
Pandas: 0.70s | qutePandas: 0.0039s
Transformation
rename
Speedup: 6183.1x
Pandas: 0.61s | qutePandas: 0.0001s
cast
Speedup: 6.1x
Pandas: 0.00s | qutePandas: 0.0007s
drop_col
Speedup: 5589.4x
Pandas: 0.55s | qutePandas: 0.0001s
Grouping
groupby_sum
Speedup: 83.3x
Pandas: 0.29s | qutePandas: 0.0034s
groupby_avg
Speedup: 53.1x
Pandas: 0.28s | qutePandas: 0.0053s
Joining
merge (left)
Speedup: 46.7727x
Pandas: 0.8145s | qutePandas: 0.0174s
merge (inner)
Speedup: 10.9496x
Pandas: 0.7914s | qutePandas: 0.0723s
merge (right)
Speedup: 51.4244x
Pandas: 3.0659s | qutePandas: 0.0596s
merge (outer)
Speedup: 25.9505x
Pandas: 3.0630s | qutePandas: 0.1180s
I/O
from_csv
Speedup: 1.3881x
Pandas: 0.7469s | qutePandas: 0.5381s
to_csv
Speedup: 2.9667x
Pandas: 4.7015s | qutePandas: 1.5848s
Apply
apply (row-wise)
Speedup: 47.7924x
Pandas: 0.51s | qutePandas: 0.9419s
Introspection
dtypes
Speedup: 0.9x
Pandas: 0.00s | qutePandas: 0.0001s