6

Recently i got the following error while resolving a VERY large number of promises:

RangeError: Too many elements passed to Promise.all

I couldn't find any information regarding limits on MDN or ECMA-262.

  • 1
    100.000 @loretoparisi – Nick LeBlanc Apr 18 at 21:13
  • 1
    Let's investigate this! – loretoparisi Apr 18 at 21:20
  • 1
    I cannot reproduce this. Both in Chrome and Firefox I could pass an array with 500000 elements, and got no error. Specific to nodejs? – trincot Apr 18 at 21:23
  • 2
    Isn't it based on your own system's memory? – RichS Apr 18 at 21:34
  • 1
    So the answer it is yes and the code it is here: github.com/v8/v8/blob/master/src/builtins/… More info in the answer below! – loretoparisi Apr 18 at 21:53
6

According to the V8/V8 error code TooManyElementsInPromiseAll of the source code objects Promise

  T(TooManyElementsInPromiseAll, "Too many elements passed to Promise.all")

there is this limit. For the Promise.all i.e. the C++ PromiseAll we have there is a concept of MaximumFunctionContextSlots and kPromiseAllResolveElementCapabilitySlot, here it is the most interesting stuff from the source code:

// TODO(bmeurer): Move this to a proper context map in contexts.h?
  // Similar to the AwaitContext that we introduced for await closures.
  enum PromiseAllResolveElementContextSlots {
    // Remaining elements count
    kPromiseAllResolveElementRemainingSlot = Context::MIN_CONTEXT_SLOTS,

    // Promise capability from Promise.all
    kPromiseAllResolveElementCapabilitySlot,

    // Values array from Promise.all
    kPromiseAllResolveElementValuesArraySlot,

    kPromiseAllResolveElementLength
  };

I would expect to see a error throw like here

ThrowTypeError(context, MessageTemplate::TooManyElementsInPromiseAll);

Here it is the code that raise the TooManyElementsInPromiseAll error. Thank to Clarence that pointed me in the right direction!

BIND(&too_many_elements);
  {
    // If there are too many elements (currently more than 2**21-1), raise a
    // RangeError here (which is caught directly and turned into a rejection)
    // of the resulting promise. We could gracefully handle this case as well
    // and support more than this number of elements by going to a separate
    // function and pass the larger indices via a separate context, but it
    // doesn't seem likely that we need this, and it's unclear how the rest
    // of the system deals with 2**21 live Promises anyways.
    Node* const result =
        CallRuntime(Runtime::kThrowRangeError, native_context,
                    SmiConstant(MessageTemplate::kTooManyElementsInPromiseAll));
    GotoIfException(result, &close_iterator, var_exception);
    Unreachable();
  }

The check of this limit it is here

// Check if we reached the limit.
    TNode<Smi> const index = var_index.value();
    GotoIf(SmiEqual(index, SmiConstant(PropertyArray::HashField::kMax)),
           &too_many_elements);

so the kMax should solve the clue!

  • 2
    And yet another piece to the puzzle is in: contexts.h... where we can see NATIVE_CONTEXT_INTRINSIC_FUNCTIONS (heap-allocated activation contexts), including MIN_CONTEXT_SLOTS, which is the base value for the PromiseAllResolveElementContextSlots enum – RichS Apr 18 at 22:00
  • 1
    kMax can be any "max constant". The lowercase k in C++ is a naming convention used to specify a constant. – RichS Apr 18 at 22:11
  • 1
    "SMI" == "Static Memory Interface". Looking up the kMax field in the SmiConstant hashmap contains the answer. Each user will probably experience different limits based on system memory. Remove a RAM stick from your system and repeat the test.. You'll see what I mean :) – RichS Apr 18 at 22:23
  • 1
    ...where SmiConstant = Type::NewConstant(js_heap_broker(), smi, zone);! – loretoparisi Apr 18 at 22:42
2

From the V8 unit tests, we see this:

// Make sure we properly throw a RangeError when overflowing the maximum
// number of elements for Promise.all, which is capped at 2^21 bits right
// now, since we store the indices as identity hash on the resolve element
// closures.
const a = new Array(2 ** 21 - 1);
const p = Promise.resolve(1);
for (let i = 0; i < a.length; ++i) a[i] = p;
testAsync(assert => {
  assert.plan(1);
  Promise.all(a).then(assert.unreachable, reason => {
    assert.equals(true, reason instanceof RangeError);
  });
});

It looks like the maximum number of elements is capped at 2^21 (= 2097151), which is in line the practical tests that the other answers ran.

  • 2
    good catch! I'm still searching where the TooManyElementsInPromiseAll is raised, but I do not find it! – loretoparisi Apr 18 at 21:47
  • 2
    My follow-up question would be: what may be the reason for capping it at 2^21? – RichS Apr 18 at 21:48
2

I can say what the limit appears to be, though I can't pinpoint why exactly it is the way it is in the V8 source code. I wrote the following code (only run it if you're bored, it'll take a while):

if (!window.chrome) {
  throw new Error('Only try this in Chromium');
}

// somewhere between 1e6 and 1e7
let testAmountStart = 5.5e6;
let changeBy = 4.5e6;
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const next = (testAmount) => {
  changeBy = Math.ceil(changeBy / 2);
  if (changeBy === 1) {
    console.log('done');
    return;
  }
  console.log('start ' + testAmount);
  const proms = new Array(testAmount).fill(undefined);
  Promise.all(proms)
    .then(() => {
      // make this loop not fully blocking
      // give time for garbage collection
      console.log(testAmount + ': OK');
      delay(100).then(() => next(testAmount + changeBy));
    }).catch((e) => {
      console.log(testAmount + ': ' + e.message);
      delay(100).then(() => next(testAmount - changeBy));
    });
};
next(testAmountStart);

The result: an error is thrown when an array with 2097151 elements is passed, but 2097150 elements is OK:

const tryProms = length => {
  const proms = new Array(length).fill(undefined);
  Promise.all(proms)
      .then(() => {
      console.log('ok ' + length);
    }).catch(() => {
      console.log('error ' + length);
    });
};
tryProms(2097150);
tryProms(2097151);

So, 2097150 is the limit. It likely has something to do with the fact that 2097151 is 0x1FFFFF.

  • Just to clarify, are you getting the exact same error as the OP? "RangeError" – RichS Apr 18 at 21:38
  • 3
    Yes, that's the error, I added a log for the error message – CertainPerformance Apr 18 at 21:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.