Repository navigation
Description
lpad and rpad panic or abort on a negative or very large length in an array of lengths
Describe the bug
When the length argument is an array, lpad and rpad estimate the output size before they validate any length. pad_data_capacity adds up len as usize over every row. A negative length wraps to a value near usize::MAX, and large positive lengths can add up to more than any allocation can satisfy. The builder's with_capacity then panics with capacity overflow, or the allocation fails and the process aborts. With lpad on an array of "a" strings:
| Lengths | Result |
|---|---|
[-1] |
panics: capacity overflow |
[3_000_000_000] |
Err: lpad requested length 3000000000 too large, maximum allowed length is 2147483647 |
[1 << 62] |
aborts: memory allocation of 4611686018427387904 bytes failed |
[i64::MAX] |
aborts: memory allocation of 9223372036854775807 bytes failed |
[i64::MAX / 2, i64::MAX / 2 + 1] |
aborts: memory allocation of 9223372036854775807 bytes failed |
[i64::MAX, i64::MAX] |
panics: capacity overflow |
With a scalar length, parse_target_len treats a negative length as 0, as its comment says, and returns the "requested length too large" error above i32::MAX. So the same lengths as scalars return "" or that error. The sqllogictests expect "" for lpad('hi', -1) and rpad('hi', -1), but they pass the length as a literal, so they take the scalar path. The tests that read the length from a column use neither negative nor very large lengths.
pad_data_capacity came in with #23586, and #26367 moved it into the pad kernel that lpad and rpad share. #3829 fixed an earlier panic on a negative length.
To Reproduce
On main at 926901f:
use std::sync::Arc;
use arrow::array::{ArrayRef, Int64Array, StringArray};
use arrow::datatypes::Field;
use datafusion_common::config::ConfigOptions;
use datafusion_expr::{ColumnarValue, ReturnFieldArgs, ScalarFunctionArgs};
use datafusion_functions::unicode::lpad;
let udf = lpad();
let args = vec![
ColumnarValue::Array(Arc::new(StringArray::from(vec!["a"])) as ArrayRef),
// As ColumnarValue::Scalar(ScalarValue::Int64(Some(-1))), the result is [""]
// With vec![1 << 62], the process aborts instead
ColumnarValue::Array(Arc::new(Int64Array::from(vec![-1])) as ArrayRef),
];
let arg_fields: Vec<_> = args
.iter()
.map(|a| Arc::new(Field::new("a", a.data_type(), true)))
.collect();
let return_field = udf
.return_field_from_args(ReturnFieldArgs {
arg_fields: &arg_fields,
scalar_arguments: &[None, None],
})
.unwrap();
let result = udf.invoke_with_args(ScalarFunctionArgs {
args,
arg_fields,
number_rows: 1,
return_field,
config_options: Arc::new(ConfigOptions::default()),
});
// panics: capacity overflowrpad panics the same way on -1.
Expected behavior
The same results as with a scalar length: [""] for a negative length, and the "requested length too large" error for a length above i32::MAX. PostgreSQL also treats a negative length as 0 (oracle_compat.c), and its regression test expects '' for lpad('hi', -5, 'xy'). Spark's lpad returns an empty string for any length of 0 or less.
Additional context
Found by property-based tests that call each function with its arguments as arrays and as scalars, and compare the results.
Willingness to contribute
I cannot contribute a fix for this bug at this time
Activity
[-]`lpad` and `rpad` panic with "capacity overflow" on a negative length in an array of lengths[/-][+]`lpad` and `rpad` panic or abort on a negative or very large length in an array of lengths[/+]gonappuccino commentedon Oct 30, 2026
take