Skip to content

lpad and rpad panic or abort on a negative or very large length in an array of lengths #26949

Description

@mbutrovich

lpad and rpad panic or abort on a negative or very large length in an array of lengths

Describe the bug

When the length argument is an array, lpad and rpad estimate the output size before they validate any length. pad_data_capacity adds up len as usize over every row. A negative length wraps to a value near usize::MAX, and large positive lengths can add up to more than any allocation can satisfy. The builder's with_capacity then panics with capacity overflow, or the allocation fails and the process aborts. With lpad on an array of "a" strings:

Lengths Result
[-1] panics: capacity overflow
[3_000_000_000] Err: lpad requested length 3000000000 too large, maximum allowed length is 2147483647
[1 << 62] aborts: memory allocation of 4611686018427387904 bytes failed
[i64::MAX] aborts: memory allocation of 9223372036854775807 bytes failed
[i64::MAX / 2, i64::MAX / 2 + 1] aborts: memory allocation of 9223372036854775807 bytes failed
[i64::MAX, i64::MAX] panics: capacity overflow

With a scalar length, parse_target_len treats a negative length as 0, as its comment says, and returns the "requested length too large" error above i32::MAX. So the same lengths as scalars return "" or that error. The sqllogictests expect "" for lpad('hi', -1) and rpad('hi', -1), but they pass the length as a literal, so they take the scalar path. The tests that read the length from a column use neither negative nor very large lengths.

pad_data_capacity came in with #23586, and #26367 moved it into the pad kernel that lpad and rpad share. #3829 fixed an earlier panic on a negative length.

To Reproduce

On main at 926901f:

use std::sync::Arc;

use arrow::array::{ArrayRef, Int64Array, StringArray};
use arrow::datatypes::Field;
use datafusion_common::config::ConfigOptions;
use datafusion_expr::{ColumnarValue, ReturnFieldArgs, ScalarFunctionArgs};
use datafusion_functions::unicode::lpad;

let udf = lpad();
let args = vec![
    ColumnarValue::Array(Arc::new(StringArray::from(vec!["a"])) as ArrayRef),
    // As ColumnarValue::Scalar(ScalarValue::Int64(Some(-1))), the result is [""]
    // With vec![1 << 62], the process aborts instead
    ColumnarValue::Array(Arc::new(Int64Array::from(vec![-1])) as ArrayRef),
];
let arg_fields: Vec<_> = args
    .iter()
    .map(|a| Arc::new(Field::new("a", a.data_type(), true)))
    .collect();
let return_field = udf
    .return_field_from_args(ReturnFieldArgs {
        arg_fields: &arg_fields,
        scalar_arguments: &[None, None],
    })
    .unwrap();
let result = udf.invoke_with_args(ScalarFunctionArgs {
    args,
    arg_fields,
    number_rows: 1,
    return_field,
    config_options: Arc::new(ConfigOptions::default()),
});
// panics: capacity overflow

rpad panics the same way on -1.

Expected behavior

The same results as with a scalar length: [""] for a negative length, and the "requested length too large" error for a length above i32::MAX. PostgreSQL also treats a negative length as 0 (oracle_compat.c), and its regression test expects '' for lpad('hi', -5, 'xy'). Spark's lpad returns an empty string for any length of 0 or less.

Additional context

Found by property-based tests that call each function with its arguments as arrays and as scalars, and compare the results.

Willingness to contribute

I cannot contribute a fix for this bug at this time

Activity

  1. added theissue type on Oct 30, 2026
  2. changed the title [-]`lpad` and `rpad` panic with "capacity overflow" on a negative length in an array of lengths[/-] [+]`lpad` and `rpad` panic or abort on a negative or very large length in an array of lengths[/+] on Oct 30, 2026
  3. gonappuccino commented on Oct 30, 2026

    @gonappuccino

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggood first issueGood for newcomers

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions