Skip to content

Commit df37c10

Browse files
jasnellMylesBorins
jasnell
authored and
MylesBorins
committedAug 31, 2021
buffer: introduce Blob
The `Blob` object is an immutable data buffer. This is a first step towards alignment with the `Blob` Web API. Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #36811 Backport-PR-URL: #39704 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
·
v14.21.3v14.18.0
1 parent 223494c commit df37c10

File tree

13 files changed

+1014
-0
lines changed

13 files changed

+1014
-0
lines changed
 

Diff for: ‎doc/api/buffer.md

+114
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,119 @@ for (const b of buf) {
287287
Additionally, the [`buf.values()`][], [`buf.keys()`][], and
288288
[`buf.entries()`][] methods can be used to create iterators.
289289

290+
## Class: `Blob`
291+
<!-- YAML
292+
added: REPLACEME
293+
-->
294+
295+
> Stability: 1 - Experimental
296+
297+
A [`Blob`][] encapsulates immutable, raw data that can be safely shared across
298+
multiple worker threads.
299+
300+
### `new buffer.Blob([sources[, options]])`
301+
<!-- YAML
302+
added: REPLACEME
303+
-->
304+
305+
* `sources` {string[]|ArrayBuffer[]|TypedArray[]|DataView[]|Blob[]} An array
306+
of string, {ArrayBuffer}, {TypedArray}, {DataView}, or {Blob} objects, or
307+
any mix of such objects, that will be stored within the `Blob`.
308+
* `options` {Object}
309+
* `encoding` {string} The character encoding to use for string sources.
310+
**Default**: `'utf8'`.
311+
* `type` {string} The Blob content-type. The intent is for `type` to convey
312+
the MIME media type of the data, however no validation of the type format
313+
is performed.
314+
315+
Creates a new `Blob` object containing a concatenation of the given sources.
316+
317+
{ArrayBuffer}, {TypedArray}, {DataView}, and {Buffer} sources are copied into
318+
the 'Blob' and can therefore be safely modified after the 'Blob' is created.
319+
320+
String sources are also copied into the `Blob`.
321+
322+
### `blob.arrayBuffer()`
323+
<!-- YAML
324+
added: REPLACEME
325+
-->
326+
327+
* Returns: {Promise}
328+
329+
Returns a promise that fulfills with an {ArrayBuffer} containing a copy of
330+
the `Blob` data.
331+
332+
### `blob.size`
333+
<!-- YAML
334+
added: REPLACEME
335+
-->
336+
337+
The total size of the `Blob` in bytes.
338+
339+
### `blob.slice([start, [end, [type]]])`
340+
<!-- YAML
341+
added: REPLACEME
342+
-->
343+
344+
* `start` {number} The starting index.
345+
* `end` {number} The ending index.
346+
* `type` {string} The content-type for the new `Blob`
347+
348+
Creates and returns a new `Blob` containing a subset of this `Blob` objects
349+
data. The original `Blob` is not alterered.
350+
351+
### `blob.text()`
352+
<!-- YAML
353+
added: REPLACEME
354+
-->
355+
356+
* Returns: {Promise}
357+
358+
Returns a promise that resolves the contents of the `Blob` decoded as a UTF-8
359+
string.
360+
361+
### `blob.type`
362+
<!-- YAML
363+
added: REPLACEME
364+
-->
365+
366+
* Type: {string}
367+
368+
The content-type of the `Blob`.
369+
370+
### `Blob` objects and `MessageChannel`
371+
372+
Once a {Blob} object is created, it can be sent via `MessagePort` to multiple
373+
destinations without transfering or immediately copying the data. The data
374+
contained by the `Blob` is copied only when the `arrayBuffer()` or `text()`
375+
methods are called.
376+
377+
```js
378+
const { Blob } = require('buffer');
379+
const blob = new Blob(['hello there']);
380+
const { setTimeout: delay } = require('timers/promises');
381+
382+
const mc1 = new MessageChannel();
383+
const mc2 = new MessageChannel();
384+
385+
mc1.port1.onmessage = async ({ data }) => {
386+
console.log(await data.arrayBuffer());
387+
mc1.port1.close();
388+
};
389+
390+
mc2.port1.onmessage = async ({ data }) => {
391+
await delay(1000);
392+
console.log(await data.arrayBuffer());
393+
mc2.port1.close();
394+
};
395+
396+
mc1.port2.postMessage(blob);
397+
mc2.port2.postMessage(blob);
398+
399+
// The Blob is still usable after posting.
400+
data.text().then(console.log);
401+
```
402+
290403
## Class: `Buffer`
291404

292405
The `Buffer` class is a global type for dealing with binary data directly.
@@ -3397,6 +3510,7 @@ introducing security vulnerabilities into an application.
33973510
[UTF-8]: https://en.wikipedia.org/wiki/UTF-8
33983511
[WHATWG Encoding Standard]: https://encoding.spec.whatwg.org/
33993512
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
3513+
[`Blob`]: https://developer.mozilla.org/en-US/docs/Web/API/Blob
34003514
[`Buffer.alloc()`]: #buffer_static_method_buffer_alloc_size_fill_encoding
34013515
[`Buffer.allocUnsafe()`]: #buffer_static_method_buffer_allocunsafe_size
34023516
[`Buffer.allocUnsafeSlow()`]: #buffer_static_method_buffer_allocunsafeslow_size

Diff for: ‎lib/buffer.js

+5
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,10 @@ const {
116116
addBufferPrototypeMethods
117117
} = require('internal/buffer');
118118

119+
const {
120+
Blob,
121+
} = require('internal/blob');
122+
119123
FastBuffer.prototype.constructor = Buffer;
120124
Buffer.prototype = FastBuffer.prototype;
121125
addBufferPrototypeMethods(Buffer.prototype);
@@ -1259,6 +1263,7 @@ function atob(input) {
12591263
}
12601264

12611265
module.exports = {
1266+
Blob,
12621267
Buffer,
12631268
SlowBuffer,
12641269
transcode,

Diff for: ‎lib/internal/blob.js

+240
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
'use strict';
2+
3+
const {
4+
ArrayFrom,
5+
ObjectSetPrototypeOf,
6+
Promise,
7+
PromiseResolve,
8+
RegExpPrototypeTest,
9+
StringPrototypeToLowerCase,
10+
Symbol,
11+
SymbolIterator,
12+
Uint8Array,
13+
} = primordials;
14+
15+
const {
16+
createBlob,
17+
FixedSizeBlobCopyJob,
18+
} = internalBinding('buffer');
19+
20+
const { TextDecoder } = require('internal/encoding');
21+
22+
const {
23+
JSTransferable,
24+
kClone,
25+
kDeserialize,
26+
} = require('internal/worker/js_transferable');
27+
28+
const {
29+
isAnyArrayBuffer,
30+
isArrayBufferView,
31+
} = require('internal/util/types');
32+
33+
const {
34+
customInspectSymbol: kInspect,
35+
emitExperimentalWarning,
36+
} = require('internal/util');
37+
const { inspect } = require('internal/util/inspect');
38+
39+
const {
40+
AbortError,
41+
codes: {
42+
ERR_INVALID_ARG_TYPE,
43+
ERR_BUFFER_TOO_LARGE,
44+
ERR_OUT_OF_RANGE,
45+
}
46+
} = require('internal/errors');
47+
48+
const {
49+
validateObject,
50+
validateString,
51+
validateUint32,
52+
isUint32,
53+
} = require('internal/validators');
54+
55+
const kHandle = Symbol('kHandle');
56+
const kType = Symbol('kType');
57+
const kLength = Symbol('kLength');
58+
59+
let Buffer;
60+
61+
function deferred() {
62+
let res, rej;
63+
const promise = new Promise((resolve, reject) => {
64+
res = resolve;
65+
rej = reject;
66+
});
67+
return { promise, resolve: res, reject: rej };
68+
}
69+
70+
function lazyBuffer() {
71+
if (Buffer === undefined)
72+
Buffer = require('buffer').Buffer;
73+
return Buffer;
74+
}
75+
76+
function isBlob(object) {
77+
return object?.[kHandle] !== undefined;
78+
}
79+
80+
function getSource(source, encoding) {
81+
if (isBlob(source))
82+
return [source.size, source[kHandle]];
83+
84+
if (typeof source === 'string') {
85+
source = lazyBuffer().from(source, encoding);
86+
} else if (isAnyArrayBuffer(source)) {
87+
source = new Uint8Array(source);
88+
} else if (!isArrayBufferView(source)) {
89+
throw new ERR_INVALID_ARG_TYPE(
90+
'source',
91+
[
92+
'string',
93+
'ArrayBuffer',
94+
'SharedArrayBuffer',
95+
'Buffer',
96+
'TypedArray',
97+
'DataView'
98+
],
99+
source);
100+
}
101+
102+
// We copy into a new Uint8Array because the underlying
103+
// BackingStores are going to be detached and owned by
104+
// the Blob. We also don't want to have to worry about
105+
// byte offsets.
106+
source = new Uint8Array(source);
107+
return [source.byteLength, source];
108+
}
109+
110+
class InternalBlob extends JSTransferable {
111+
constructor(handle, length, type = '') {
112+
super();
113+
this[kHandle] = handle;
114+
this[kType] = type;
115+
this[kLength] = length;
116+
}
117+
}
118+
119+
class Blob extends JSTransferable {
120+
constructor(sources = [], options) {
121+
emitExperimentalWarning('buffer.Blob');
122+
if (sources === null ||
123+
typeof sources[SymbolIterator] !== 'function' ||
124+
typeof sources === 'string') {
125+
throw new ERR_INVALID_ARG_TYPE('sources', 'Iterable', sources);
126+
}
127+
if (options !== undefined)
128+
validateObject(options, 'options');
129+
const {
130+
encoding = 'utf8',
131+
type = '',
132+
} = { ...options };
133+
134+
let length = 0;
135+
const sources_ = ArrayFrom(sources, (source) => {
136+
const { 0: len, 1: src } = getSource(source, encoding);
137+
length += len;
138+
return src;
139+
});
140+
141+
// This is a MIME media type but we're not actively checking the syntax.
142+
// But, to be fair, neither does Chrome.
143+
validateString(type, 'options.type');
144+
145+
if (!isUint32(length))
146+
throw new ERR_BUFFER_TOO_LARGE(0xFFFFFFFF);
147+
148+
super();
149+
this[kHandle] = createBlob(sources_, length);
150+
this[kLength] = length;
151+
this[kType] = RegExpPrototypeTest(/[^\u{0020}-\u{007E}]/u, type) ?
152+
'' : StringPrototypeToLowerCase(type);
153+
}
154+
155+
[kInspect](depth, options) {
156+
if (depth < 0)
157+
return this;
158+
159+
const opts = {
160+
...options,
161+
depth: options.depth == null ? null : options.depth - 1
162+
};
163+
164+
return `Blob ${inspect({
165+
size: this.size,
166+
type: this.type,
167+
}, opts)}`;
168+
}
169+
170+
[kClone]() {
171+
const handle = this[kHandle];
172+
const type = this[kType];
173+
const length = this[kLength];
174+
return {
175+
data: { handle, type, length },
176+
deserializeInfo: 'internal/blob:InternalBlob'
177+
};
178+
}
179+
180+
[kDeserialize]({ handle, type, length }) {
181+
this[kHandle] = handle;
182+
this[kType] = type;
183+
this[kLength] = length;
184+
}
185+
186+
get type() { return this[kType]; }
187+
188+
get size() { return this[kLength]; }
189+
190+
slice(start = 0, end = (this[kLength]), type = this[kType]) {
191+
validateUint32(start, 'start');
192+
if (end < 0) end = this[kLength] + end;
193+
validateUint32(end, 'end');
194+
validateString(type, 'type');
195+
if (end < start)
196+
throw new ERR_OUT_OF_RANGE('end', 'greater than start', end);
197+
if (end > this[kLength])
198+
throw new ERR_OUT_OF_RANGE('end', 'less than or equal to length', end);
199+
return new InternalBlob(
200+
this[kHandle].slice(start, end),
201+
end - start, type);
202+
}
203+
204+
async arrayBuffer() {
205+
const job = new FixedSizeBlobCopyJob(this[kHandle]);
206+
207+
const ret = job.run();
208+
if (ret !== undefined)
209+
return PromiseResolve(ret);
210+
211+
const {
212+
promise,
213+
resolve,
214+
reject
215+
} = deferred();
216+
job.ondone = (err, ab) => {
217+
if (err !== undefined)
218+
return reject(new AbortError());
219+
resolve(ab);
220+
};
221+
222+
return promise;
223+
}
224+
225+
async text() {
226+
const dec = new TextDecoder();
227+
return dec.decode(await this.arrayBuffer());
228+
}
229+
}
230+
231+
InternalBlob.prototype.constructor = Blob;
232+
ObjectSetPrototypeOf(
233+
InternalBlob.prototype,
234+
Blob.prototype);
235+
236+
module.exports = {
237+
Blob,
238+
InternalBlob,
239+
isBlob,
240+
};

Diff for: ‎node.gyp

+3
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@
101101
'lib/internal/assert/assertion_error.js',
102102
'lib/internal/assert/calltracker.js',
103103
'lib/internal/async_hooks.js',
104+
'lib/internal/blob.js',
104105
'lib/internal/blocklist.js',
105106
'lib/internal/buffer.js',
106107
'lib/internal/cli_table.js',
@@ -589,6 +590,7 @@
589590
'src/node.cc',
590591
'src/node_api.cc',
591592
'src/node_binding.cc',
593+
'src/node_blob.cc',
592594
'src/node_buffer.cc',
593595
'src/node_config.cc',
594596
'src/node_constants.cc',
@@ -687,6 +689,7 @@
687689
'src/node_api.h',
688690
'src/node_api_types.h',
689691
'src/node_binding.h',
692+
'src/node_blob.h',
690693
'src/node_buffer.h',
691694
'src/node_constants.h',
692695
'src/node_context_data.h',

Diff for: ‎src/async_wrap.h

+1
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ namespace node {
3838
V(ELDHISTOGRAM) \
3939
V(FILEHANDLE) \
4040
V(FILEHANDLECLOSEREQ) \
41+
V(FIXEDSIZEBLOBCOPY) \
4142
V(FSEVENTWRAP) \
4243
V(FSREQCALLBACK) \
4344
V(FSREQPROMISE) \

Diff for: ‎src/env.h

+1
Original file line numberDiff line numberDiff line change
@@ -413,6 +413,7 @@ constexpr size_t kFsStatsBufferLength =
413413
V(async_wrap_object_ctor_template, v8::FunctionTemplate) \
414414
V(base_object_ctor_template, v8::FunctionTemplate) \
415415
V(binding_data_ctor_template, v8::FunctionTemplate) \
416+
V(blob_constructor_template, v8::FunctionTemplate) \
416417
V(blocklist_constructor_template, v8::FunctionTemplate) \
417418
V(compiled_fn_entry_template, v8::ObjectTemplate) \
418419
V(dir_instance_template, v8::ObjectTemplate) \

Diff for: ‎src/node_blob.cc

+323
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,323 @@
1+
#include "node_blob.h"
2+
#include "async_wrap-inl.h"
3+
#include "base_object-inl.h"
4+
#include "env-inl.h"
5+
#include "memory_tracker-inl.h"
6+
#include "node_errors.h"
7+
#include "threadpoolwork-inl.h"
8+
#include "v8.h"
9+
10+
#include <algorithm>
11+
12+
namespace node {
13+
14+
using v8::Array;
15+
using v8::ArrayBuffer;
16+
using v8::ArrayBufferView;
17+
using v8::BackingStore;
18+
using v8::Context;
19+
using v8::EscapableHandleScope;
20+
using v8::Function;
21+
using v8::FunctionCallbackInfo;
22+
using v8::FunctionTemplate;
23+
using v8::HandleScope;
24+
using v8::Local;
25+
using v8::MaybeLocal;
26+
using v8::Number;
27+
using v8::Object;
28+
using v8::Uint32;
29+
using v8::Undefined;
30+
using v8::Value;
31+
32+
void Blob::Initialize(Environment* env, v8::Local<v8::Object> target) {
33+
env->SetMethod(target, "createBlob", New);
34+
FixedSizeBlobCopyJob::Initialize(env, target);
35+
}
36+
37+
Local<FunctionTemplate> Blob::GetConstructorTemplate(Environment* env) {
38+
Local<FunctionTemplate> tmpl = env->blob_constructor_template();
39+
if (tmpl.IsEmpty()) {
40+
tmpl = FunctionTemplate::New(env->isolate());
41+
tmpl->InstanceTemplate()->SetInternalFieldCount(1);
42+
tmpl->Inherit(BaseObject::GetConstructorTemplate(env));
43+
tmpl->SetClassName(
44+
FIXED_ONE_BYTE_STRING(env->isolate(), "Blob"));
45+
env->SetProtoMethod(tmpl, "toArrayBuffer", ToArrayBuffer);
46+
env->SetProtoMethod(tmpl, "slice", ToSlice);
47+
env->set_blob_constructor_template(tmpl);
48+
}
49+
return tmpl;
50+
}
51+
52+
bool Blob::HasInstance(Environment* env, v8::Local<v8::Value> object) {
53+
return GetConstructorTemplate(env)->HasInstance(object);
54+
}
55+
56+
BaseObjectPtr<Blob> Blob::Create(
57+
Environment* env,
58+
const std::vector<BlobEntry> store,
59+
size_t length) {
60+
61+
HandleScope scope(env->isolate());
62+
63+
Local<Function> ctor;
64+
if (!GetConstructorTemplate(env)->GetFunction(env->context()).ToLocal(&ctor))
65+
return BaseObjectPtr<Blob>();
66+
67+
Local<Object> obj;
68+
if (!ctor->NewInstance(env->context()).ToLocal(&obj))
69+
return BaseObjectPtr<Blob>();
70+
71+
return MakeBaseObject<Blob>(env, obj, store, length);
72+
}
73+
74+
void Blob::New(const FunctionCallbackInfo<Value>& args) {
75+
Environment* env = Environment::GetCurrent(args);
76+
CHECK(args[0]->IsArray()); // sources
77+
CHECK(args[1]->IsUint32()); // length
78+
79+
std::vector<BlobEntry> entries;
80+
81+
size_t length = args[1].As<Uint32>()->Value();
82+
size_t len = 0;
83+
Local<Array> ary = args[0].As<Array>();
84+
for (size_t n = 0; n < ary->Length(); n++) {
85+
Local<Value> entry;
86+
if (!ary->Get(env->context(), n).ToLocal(&entry))
87+
return;
88+
CHECK(entry->IsArrayBufferView() || Blob::HasInstance(env, entry));
89+
if (entry->IsArrayBufferView()) {
90+
Local<ArrayBufferView> view = entry.As<ArrayBufferView>();
91+
CHECK_EQ(view->ByteOffset(), 0);
92+
std::shared_ptr<BackingStore> store = view->Buffer()->GetBackingStore();
93+
size_t byte_length = view->ByteLength();
94+
view->Buffer()->Detach(); // The Blob will own the backing store now.
95+
entries.emplace_back(BlobEntry{std::move(store), byte_length, 0});
96+
len += byte_length;
97+
} else {
98+
Blob* blob;
99+
ASSIGN_OR_RETURN_UNWRAP(&blob, entry);
100+
auto source = blob->entries();
101+
entries.insert(entries.end(), source.begin(), source.end());
102+
len += blob->length();
103+
}
104+
}
105+
CHECK_EQ(length, len);
106+
107+
BaseObjectPtr<Blob> blob = Create(env, entries, length);
108+
if (blob)
109+
args.GetReturnValue().Set(blob->object());
110+
}
111+
112+
void Blob::ToArrayBuffer(const FunctionCallbackInfo<Value>& args) {
113+
Environment* env = Environment::GetCurrent(args);
114+
Blob* blob;
115+
ASSIGN_OR_RETURN_UNWRAP(&blob, args.Holder());
116+
Local<Value> ret;
117+
if (blob->GetArrayBuffer(env).ToLocal(&ret))
118+
args.GetReturnValue().Set(ret);
119+
}
120+
121+
void Blob::ToSlice(const FunctionCallbackInfo<Value>& args) {
122+
Environment* env = Environment::GetCurrent(args);
123+
Blob* blob;
124+
ASSIGN_OR_RETURN_UNWRAP(&blob, args.Holder());
125+
CHECK(args[0]->IsUint32());
126+
CHECK(args[1]->IsUint32());
127+
size_t start = args[0].As<Uint32>()->Value();
128+
size_t end = args[1].As<Uint32>()->Value();
129+
BaseObjectPtr<Blob> slice = blob->Slice(env, start, end);
130+
if (slice)
131+
args.GetReturnValue().Set(slice->object());
132+
}
133+
134+
void Blob::MemoryInfo(MemoryTracker* tracker) const {
135+
tracker->TrackFieldWithSize("store", length_);
136+
}
137+
138+
MaybeLocal<Value> Blob::GetArrayBuffer(Environment* env) {
139+
EscapableHandleScope scope(env->isolate());
140+
size_t len = length();
141+
std::shared_ptr<BackingStore> store =
142+
ArrayBuffer::NewBackingStore(env->isolate(), len);
143+
if (len > 0) {
144+
unsigned char* dest = static_cast<unsigned char*>(store->Data());
145+
size_t total = 0;
146+
for (const auto& entry : entries()) {
147+
unsigned char* src = static_cast<unsigned char*>(entry.store->Data());
148+
src += entry.offset;
149+
memcpy(dest, src, entry.length);
150+
dest += entry.length;
151+
total += entry.length;
152+
CHECK_LE(total, len);
153+
}
154+
}
155+
156+
return scope.Escape(ArrayBuffer::New(env->isolate(), store));
157+
}
158+
159+
BaseObjectPtr<Blob> Blob::Slice(Environment* env, size_t start, size_t end) {
160+
CHECK_LE(start, length());
161+
CHECK_LE(end, length());
162+
CHECK_LE(start, end);
163+
164+
std::vector<BlobEntry> slices;
165+
size_t total = end - start;
166+
size_t remaining = total;
167+
168+
if (total == 0) return Create(env, slices, 0);
169+
170+
for (const auto& entry : entries()) {
171+
if (start + entry.offset > entry.store->ByteLength()) {
172+
start -= entry.length;
173+
continue;
174+
}
175+
176+
size_t offset = entry.offset + start;
177+
size_t len = std::min(remaining, entry.store->ByteLength() - offset);
178+
slices.emplace_back(BlobEntry{entry.store, len, offset});
179+
180+
remaining -= len;
181+
start = 0;
182+
183+
if (remaining == 0)
184+
break;
185+
}
186+
187+
return Create(env, slices, total);
188+
}
189+
190+
Blob::Blob(
191+
Environment* env,
192+
v8::Local<v8::Object> obj,
193+
const std::vector<BlobEntry>& store,
194+
size_t length)
195+
: BaseObject(env, obj),
196+
store_(store),
197+
length_(length) {
198+
MakeWeak();
199+
}
200+
201+
BaseObjectPtr<BaseObject>
202+
Blob::BlobTransferData::Deserialize(
203+
Environment* env,
204+
Local<Context> context,
205+
std::unique_ptr<worker::TransferData> self) {
206+
if (context != env->context()) {
207+
THROW_ERR_MESSAGE_TARGET_CONTEXT_UNAVAILABLE(env);
208+
return {};
209+
}
210+
return Blob::Create(env, store_, length_);
211+
}
212+
213+
BaseObject::TransferMode Blob::GetTransferMode() const {
214+
return BaseObject::TransferMode::kCloneable;
215+
}
216+
217+
std::unique_ptr<worker::TransferData> Blob::CloneForMessaging() const {
218+
return std::make_unique<BlobTransferData>(store_, length_);
219+
}
220+
221+
FixedSizeBlobCopyJob::FixedSizeBlobCopyJob(
222+
Environment* env,
223+
Local<Object> object,
224+
Blob* blob,
225+
FixedSizeBlobCopyJob::Mode mode)
226+
: AsyncWrap(env, object, AsyncWrap::PROVIDER_FIXEDSIZEBLOBCOPY),
227+
ThreadPoolWork(env),
228+
mode_(mode) {
229+
if (mode == FixedSizeBlobCopyJob::Mode::SYNC) MakeWeak();
230+
source_ = blob->entries();
231+
length_ = blob->length();
232+
}
233+
234+
void FixedSizeBlobCopyJob::AfterThreadPoolWork(int status) {
235+
Environment* env = AsyncWrap::env();
236+
CHECK_EQ(mode_, Mode::ASYNC);
237+
CHECK(status == 0 || status == UV_ECANCELED);
238+
std::unique_ptr<FixedSizeBlobCopyJob> ptr(this);
239+
HandleScope handle_scope(env->isolate());
240+
Context::Scope context_scope(env->context());
241+
Local<Value> args[2];
242+
243+
if (status == UV_ECANCELED) {
244+
args[0] = Number::New(env->isolate(), status),
245+
args[1] = Undefined(env->isolate());
246+
} else {
247+
args[0] = Undefined(env->isolate());
248+
args[1] = ArrayBuffer::New(env->isolate(), destination_);
249+
}
250+
251+
ptr->MakeCallback(env->ondone_string(), arraysize(args), args);
252+
}
253+
254+
void FixedSizeBlobCopyJob::DoThreadPoolWork() {
255+
Environment* env = AsyncWrap::env();
256+
destination_ = ArrayBuffer::NewBackingStore(env->isolate(), length_);
257+
unsigned char* dest = static_cast<unsigned char*>(destination_->Data());
258+
if (length_ > 0) {
259+
size_t total = 0;
260+
for (const auto& entry : source_) {
261+
unsigned char* src = static_cast<unsigned char*>(entry.store->Data());
262+
src += entry.offset;
263+
memcpy(dest, src, entry.length);
264+
dest += entry.length;
265+
total += entry.length;
266+
CHECK_LE(total, length_);
267+
}
268+
}
269+
}
270+
271+
void FixedSizeBlobCopyJob::MemoryInfo(MemoryTracker* tracker) const {
272+
tracker->TrackFieldWithSize("source", length_);
273+
tracker->TrackFieldWithSize(
274+
"destination",
275+
destination_ ? destination_->ByteLength() : 0);
276+
}
277+
278+
void FixedSizeBlobCopyJob::Initialize(Environment* env, Local<Object> target) {
279+
v8::Local<v8::FunctionTemplate> job = env->NewFunctionTemplate(New);
280+
job->Inherit(AsyncWrap::GetConstructorTemplate(env));
281+
job->InstanceTemplate()->SetInternalFieldCount(
282+
AsyncWrap::kInternalFieldCount);
283+
env->SetProtoMethod(job, "run", Run);
284+
env->SetConstructorFunction(target, "FixedSizeBlobCopyJob", job);
285+
}
286+
287+
void FixedSizeBlobCopyJob::New(const FunctionCallbackInfo<Value>& args) {
288+
static constexpr size_t kMaxSyncLength = 4096;
289+
static constexpr size_t kMaxEntryCount = 4;
290+
291+
Environment* env = Environment::GetCurrent(args);
292+
CHECK(args.IsConstructCall());
293+
CHECK(args[0]->IsObject());
294+
CHECK(Blob::HasInstance(env, args[0]));
295+
296+
Blob* blob;
297+
ASSIGN_OR_RETURN_UNWRAP(&blob, args[0]);
298+
299+
// This is a fairly arbitrary heuristic. We want to avoid deferring to
300+
// the threadpool if the amount of data being copied is small and there
301+
// aren't that many entries to copy.
302+
FixedSizeBlobCopyJob::Mode mode =
303+
(blob->length() < kMaxSyncLength &&
304+
blob->entries().size() < kMaxEntryCount) ?
305+
FixedSizeBlobCopyJob::Mode::SYNC :
306+
FixedSizeBlobCopyJob::Mode::ASYNC;
307+
308+
new FixedSizeBlobCopyJob(env, args.This(), blob, mode);
309+
}
310+
311+
void FixedSizeBlobCopyJob::Run(const FunctionCallbackInfo<Value>& args) {
312+
Environment* env = Environment::GetCurrent(args);
313+
FixedSizeBlobCopyJob* job;
314+
ASSIGN_OR_RETURN_UNWRAP(&job, args.Holder());
315+
if (job->mode() == FixedSizeBlobCopyJob::Mode::ASYNC)
316+
return job->ScheduleWork();
317+
318+
job->DoThreadPoolWork();
319+
args.GetReturnValue().Set(
320+
ArrayBuffer::New(env->isolate(), job->destination_));
321+
}
322+
323+
} // namespace node

Diff for: ‎src/node_blob.h

+133
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
#ifndef SRC_NODE_BLOB_H_
2+
#define SRC_NODE_BLOB_H_
3+
4+
#if defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
5+
6+
#include "async_wrap.h"
7+
#include "base_object.h"
8+
#include "env.h"
9+
#include "memory_tracker.h"
10+
#include "node_internals.h"
11+
#include "node_worker.h"
12+
#include "v8.h"
13+
14+
#include <vector>
15+
16+
namespace node {
17+
18+
struct BlobEntry {
19+
std::shared_ptr<v8::BackingStore> store;
20+
size_t length;
21+
size_t offset;
22+
};
23+
24+
class Blob : public BaseObject {
25+
public:
26+
static void Initialize(Environment* env, v8::Local<v8::Object> target);
27+
28+
static void New(const v8::FunctionCallbackInfo<v8::Value>& args);
29+
static void ToArrayBuffer(const v8::FunctionCallbackInfo<v8::Value>& args);
30+
static void ToSlice(const v8::FunctionCallbackInfo<v8::Value>& args);
31+
32+
static v8::Local<v8::FunctionTemplate> GetConstructorTemplate(
33+
Environment* env);
34+
35+
static BaseObjectPtr<Blob> Create(
36+
Environment* env,
37+
const std::vector<BlobEntry> store,
38+
size_t length);
39+
40+
static bool HasInstance(Environment* env, v8::Local<v8::Value> object);
41+
42+
const std::vector<BlobEntry> entries() const {
43+
return store_;
44+
}
45+
46+
void MemoryInfo(MemoryTracker* tracker) const override;
47+
SET_MEMORY_INFO_NAME(Blob);
48+
SET_SELF_SIZE(Blob);
49+
50+
// Copies the contents of the Blob into an ArrayBuffer.
51+
v8::MaybeLocal<v8::Value> GetArrayBuffer(Environment* env);
52+
53+
BaseObjectPtr<Blob> Slice(Environment* env, size_t start, size_t end);
54+
55+
inline size_t length() const { return length_; }
56+
57+
class BlobTransferData : public worker::TransferData {
58+
public:
59+
explicit BlobTransferData(
60+
const std::vector<BlobEntry>& store,
61+
size_t length)
62+
: store_(store),
63+
length_(length) {}
64+
65+
BaseObjectPtr<BaseObject> Deserialize(
66+
Environment* env,
67+
v8::Local<v8::Context> context,
68+
std::unique_ptr<worker::TransferData> self) override;
69+
70+
SET_MEMORY_INFO_NAME(BlobTransferData)
71+
SET_SELF_SIZE(BlobTransferData)
72+
SET_NO_MEMORY_INFO()
73+
74+
private:
75+
std::vector<BlobEntry> store_;
76+
size_t length_ = 0;
77+
};
78+
79+
BaseObject::TransferMode GetTransferMode() const override;
80+
std::unique_ptr<worker::TransferData> CloneForMessaging() const override;
81+
82+
Blob(
83+
Environment* env,
84+
v8::Local<v8::Object> obj,
85+
const std::vector<BlobEntry>& store,
86+
size_t length);
87+
88+
private:
89+
std::vector<BlobEntry> store_;
90+
size_t length_ = 0;
91+
};
92+
93+
class FixedSizeBlobCopyJob : public AsyncWrap, public ThreadPoolWork {
94+
public:
95+
enum class Mode {
96+
SYNC,
97+
ASYNC
98+
};
99+
100+
static void Initialize(Environment* env, v8::Local<v8::Object> target);
101+
static void New(const v8::FunctionCallbackInfo<v8::Value>& args);
102+
static void Run(const v8::FunctionCallbackInfo<v8::Value>& args);
103+
104+
bool IsNotIndicativeOfMemoryLeakAtExit() const override {
105+
return true;
106+
}
107+
108+
void DoThreadPoolWork() override;
109+
void AfterThreadPoolWork(int status) override;
110+
111+
Mode mode() const { return mode_; }
112+
113+
void MemoryInfo(MemoryTracker* tracker) const override;
114+
SET_MEMORY_INFO_NAME(FixedSizeBlobCopyJob)
115+
SET_SELF_SIZE(FixedSizeBlobCopyJob)
116+
117+
private:
118+
FixedSizeBlobCopyJob(
119+
Environment* env,
120+
v8::Local<v8::Object> object,
121+
Blob* blob,
122+
Mode mode = Mode::ASYNC);
123+
124+
Mode mode_;
125+
std::vector<BlobEntry> source_;
126+
std::shared_ptr<v8::BackingStore> destination_;
127+
size_t length_ = 0;
128+
};
129+
130+
} // namespace node
131+
132+
#endif // defined(NODE_WANT_INTERNALS) && NODE_WANT_INTERNALS
133+
#endif // SRC_NODE_BLOB_H_

Diff for: ‎src/node_buffer.cc

+3
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
#include "node_buffer.h"
2323
#include "allocated_buffer-inl.h"
2424
#include "node.h"
25+
#include "node_blob.h"
2526
#include "node_errors.h"
2627
#include "node_internals.h"
2728

@@ -1177,6 +1178,8 @@ void Initialize(Local<Object> target,
11771178
env->SetMethod(target, "ucs2Write", StringWrite<UCS2>);
11781179
env->SetMethod(target, "utf8Write", StringWrite<UTF8>);
11791180

1181+
Blob::Initialize(env, target);
1182+
11801183
// It can be a nullptr when running inside an isolate where we
11811184
// do not own the ArrayBuffer allocator.
11821185
if (NodeArrayBufferAllocator* allocator =

Diff for: ‎test/parallel/test-blob.js

+187
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
'use strict';
2+
3+
const common = require('../common');
4+
const assert = require('assert');
5+
const { Blob } = require('buffer');
6+
const { MessageChannel } = require('worker_threads');
7+
8+
{
9+
const b = new Blob();
10+
assert.strictEqual(b.size, 0);
11+
assert.strictEqual(b.type, '');
12+
}
13+
14+
assert.throws(() => new Blob(false), {
15+
code: 'ERR_INVALID_ARG_TYPE'
16+
});
17+
18+
assert.throws(() => new Blob('hello'), {
19+
code: 'ERR_INVALID_ARG_TYPE'
20+
});
21+
22+
assert.throws(() => new Blob({}), {
23+
code: 'ERR_INVALID_ARG_TYPE'
24+
});
25+
26+
assert.throws(() => new Blob(['test', 1]), {
27+
code: 'ERR_INVALID_ARG_TYPE'
28+
});
29+
30+
{
31+
const b = new Blob([]);
32+
assert(b);
33+
assert.strictEqual(b.size, 0);
34+
assert.strictEqual(b.type, '');
35+
36+
b.arrayBuffer().then(common.mustCall((ab) => {
37+
assert.deepStrictEqual(ab, new ArrayBuffer(0));
38+
}));
39+
b.text().then(common.mustCall((text) => {
40+
assert.strictEqual(text, '');
41+
}));
42+
const c = b.slice();
43+
assert.strictEqual(c.size, 0);
44+
}
45+
46+
{
47+
assert.throws(() => new Blob([], { type: 1 }), {
48+
code: 'ERR_INVALID_ARG_TYPE'
49+
});
50+
assert.throws(() => new Blob([], { type: false }), {
51+
code: 'ERR_INVALID_ARG_TYPE'
52+
});
53+
assert.throws(() => new Blob([], { type: {} }), {
54+
code: 'ERR_INVALID_ARG_TYPE'
55+
});
56+
}
57+
58+
{
59+
const b = new Blob(['616263'], { encoding: 'hex', type: 'foo' });
60+
assert.strictEqual(b.size, 3);
61+
assert.strictEqual(b.type, 'foo');
62+
b.text().then(common.mustCall((text) => {
63+
assert.strictEqual(text, 'abc');
64+
}));
65+
}
66+
67+
{
68+
const b = new Blob([Buffer.from('abc')]);
69+
assert.strictEqual(b.size, 3);
70+
b.text().then(common.mustCall((text) => {
71+
assert.strictEqual(text, 'abc');
72+
}));
73+
}
74+
75+
{
76+
const b = new Blob([new ArrayBuffer(3)]);
77+
assert.strictEqual(b.size, 3);
78+
b.text().then(common.mustCall((text) => {
79+
assert.strictEqual(text, '\0\0\0');
80+
}));
81+
}
82+
83+
{
84+
const b = new Blob([new Uint8Array(3)]);
85+
assert.strictEqual(b.size, 3);
86+
b.text().then(common.mustCall((text) => {
87+
assert.strictEqual(text, '\0\0\0');
88+
}));
89+
}
90+
91+
{
92+
const b = new Blob([new Blob(['abc'])]);
93+
assert.strictEqual(b.size, 3);
94+
b.text().then(common.mustCall((text) => {
95+
assert.strictEqual(text, 'abc');
96+
}));
97+
}
98+
99+
{
100+
const b = new Blob(['hello', Buffer.from('world')]);
101+
assert.strictEqual(b.size, 10);
102+
b.text().then(common.mustCall((text) => {
103+
assert.strictEqual(text, 'helloworld');
104+
}));
105+
}
106+
107+
{
108+
const b = new Blob(
109+
[
110+
'h',
111+
'e',
112+
'l',
113+
'lo',
114+
Buffer.from('world')
115+
]);
116+
assert.strictEqual(b.size, 10);
117+
b.text().then(common.mustCall((text) => {
118+
assert.strictEqual(text, 'helloworld');
119+
}));
120+
}
121+
122+
{
123+
const b = new Blob(['hello', Buffer.from('world')]);
124+
assert.strictEqual(b.size, 10);
125+
assert.strictEqual(b.type, '');
126+
127+
const c = b.slice(1, -1, 'foo');
128+
assert.strictEqual(c.type, 'foo');
129+
c.text().then(common.mustCall((text) => {
130+
assert.strictEqual(text, 'elloworl');
131+
}));
132+
133+
const d = c.slice(1, -1);
134+
d.text().then(common.mustCall((text) => {
135+
assert.strictEqual(text, 'llowor');
136+
}));
137+
138+
const e = d.slice(1, -1);
139+
e.text().then(common.mustCall((text) => {
140+
assert.strictEqual(text, 'lowo');
141+
}));
142+
143+
const f = e.slice(1, -1);
144+
f.text().then(common.mustCall((text) => {
145+
assert.strictEqual(text, 'ow');
146+
}));
147+
148+
const g = f.slice(1, -1);
149+
assert.strictEqual(g.type, 'foo');
150+
g.text().then(common.mustCall((text) => {
151+
assert.strictEqual(text, '');
152+
}));
153+
154+
assert.strictEqual(b.size, 10);
155+
assert.strictEqual(b.type, '');
156+
157+
assert.throws(() => b.slice(-1, 1), {
158+
code: 'ERR_OUT_OF_RANGE'
159+
});
160+
assert.throws(() => b.slice(1, 100), {
161+
code: 'ERR_OUT_OF_RANGE'
162+
});
163+
164+
assert.throws(() => b.slice(1, 2, false), {
165+
code: 'ERR_INVALID_ARG_TYPE'
166+
});
167+
}
168+
169+
{
170+
const b = new Blob([Buffer.from('hello'), Buffer.from('world')]);
171+
const mc = new MessageChannel();
172+
mc.port1.onmessage = common.mustCall(({ data }) => {
173+
data.text().then(common.mustCall((text) => {
174+
assert.strictEqual(text, 'helloworld');
175+
}));
176+
mc.port1.close();
177+
});
178+
mc.port2.postMessage(b);
179+
b.text().then(common.mustCall((text) => {
180+
assert.strictEqual(text, 'helloworld');
181+
}));
182+
}
183+
184+
{
185+
const b = new Blob(['hello'], { type: '\x01' });
186+
assert.strictEqual(b.type, '');
187+
}

Diff for: ‎test/parallel/test-bootstrap-modules.js

+1
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,7 @@ const expectedModules = new Set([
8686
'NativeModule internal/validators',
8787
'NativeModule internal/vm/module',
8888
'NativeModule internal/worker/js_transferable',
89+
'NativeModule internal/blob',
8990
'NativeModule path',
9091
'NativeModule timers',
9192
'NativeModule url',

Diff for: ‎test/sequential/test-async-wrap-getasyncid.js

+1
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ const { getSystemErrorName } = require('util');
5454
delete providers.ELDHISTOGRAM;
5555
delete providers.SIGINTWATCHDOG;
5656
delete providers.WORKERHEAPSNAPSHOT;
57+
delete providers.FIXEDSIZEBLOBCOPY;
5758

5859
const objKeys = Object.keys(providers);
5960
if (objKeys.length > 0)

Diff for: ‎tools/doc/type-parser.mjs

+2
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ const customTypesMap = {
3838
'WebAssembly.Instance':
3939
`${jsDocPrefix}Reference/Global_Objects/WebAssembly/Instance`,
4040

41+
'Blob': 'buffer.html#buffer_class_blob',
42+
4143
'Iterable':
4244
`${jsDocPrefix}Reference/Iteration_protocols#The_iterable_protocol`,
4345
'Iterator':

0 commit comments

Comments
 (0)
Please sign in to comment.