Skip to content

feat!: drop iso-8859-1 charset support, UTF-8 only #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlessioGr merged 1 commit into from
Mar 31, 2026
Merged

feat!: drop iso-8859-1 charset support, UTF-8 only #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 32 additions & 96 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
This is a modernized, esm-only fork of [qs](https://github.com/ljharb/qs) without unnecessary polyfills. This fork is not endorsed by [Jordan Harband](https://github.com/ljharb), the lead maintainer of qs.
This is a modernized, ESM-only fork of [qs](https://github.com/ljharb/qs) without unnecessary polyfills. This fork is not endorsed by [Jordan Harband](https://github.com/ljharb), the lead maintainer of qs.

# qs-esm

Expand All @@ -8,9 +8,33 @@ Original Maintainer of qs (not of this fork): [Jordan Harband](https://github.co

The **qs** module was originally created and maintained by [TJ Holowaychuk](https://github.com/visionmedia/node-querystring).

## Changes from qs

**Forked from [qs v6.12.1](https://github.com/ljharb/qs/tree/v6.12.1).**

Changes made in this fork:

- **ESM-only** — native ES modules; no CommonJS support
- **Node 18+ required** — removed polyfills for older Node.js versions
- **Zero dependencies** — replaced the `side-channel` npm package with a native `WeakMap` for cyclic reference detection in `stringify`
- **UTF-8 only** — dropped `charset`, `charsetSentinel`, and `interpretNumericEntities`. These were for legacy browser quirks (old IE iso-8859-1 form submissions, Rails charset sentinel params) that don't apply to modern apps. Dropping them removes branching on every encode/decode call and reduces overall complexity. Custom encoder/decoder callbacks no longer receive a `charset` argument
- **Security backport** — `arrayLimit` now applies to `[]` (bracket-only) notation, closing the same gap addressed by [GHSA-w7fw-mjwx-w883](https://github.com/ljharb/qs/security/advisories/GHSA-w7fw-mjwx-w883) in upstream qs

## Migrate from qs

Install `qs-esm`, uninstall `qs` and `@types/qs`. Then, replace `import qs from 'qs';` with `import * as qs from 'qs-esm';`.
Install `qs-esm`, uninstall `qs` and `@types/qs`. Then replace:

```js
import qs from 'qs'
// or
const qs = require('qs')
```

with:

```js
import * as qs from 'qs-esm'
```

## Usage

Expand Down Expand Up @@ -171,50 +195,6 @@ assert.deepEqual(qs.parse('foo=bar&foo=baz', { duplicates: 'first' }), { foo: 'b
assert.deepEqual(qs.parse('foo=bar&foo=baz', { duplicates: 'last' }), { foo: 'baz' })
```

If you have to deal with legacy browsers or services, there's also support for decoding percent-encoded octets as iso-8859-1:

```javascript
var oldCharset = qs.parse('a=%A7', { charset: 'iso-8859-1' })
assert.deepEqual(oldCharset, { a: '§' })
```

Some services add an initial `utf8=✓` value to forms so that old Internet Explorer versions are more likely to submit the form as utf-8.
Additionally, the server can check the value against wrong encodings of the checkmark character and detect that a query string or `application/x-www-form-urlencoded` body was _not_ sent as utf-8, eg. if the form had an `accept-charset` parameter or the containing page had a different character set.

**qs** supports this mechanism via the `charsetSentinel` option.
If specified, the `utf8` parameter will be omitted from the returned object.
It will be used to switch to `iso-8859-1`/`utf-8` mode depending on how the checkmark is encoded.

**Important**: When you specify both the `charset` option and the `charsetSentinel` option, the `charset` will be overridden when the request contains a `utf8` parameter from which the actual charset can be deduced.
In that sense the `charset` will behave as the default charset rather than the authoritative charset.

```javascript
var detectedAsUtf8 = qs.parse('utf8=%E2%9C%93&a=%C3%B8', {
charset: 'iso-8859-1',
charsetSentinel: true,
})
assert.deepEqual(detectedAsUtf8, { a: 'ø' })

// Browsers encode the checkmark as ✓ when submitting as iso-8859-1:
var detectedAsIso8859_1 = qs.parse('utf8=%26%2310003%3B&a=%F8', {
charset: 'utf-8',
charsetSentinel: true,
})
assert.deepEqual(detectedAsIso8859_1, { a: 'ø' })
```

If you want to decode the `&#...;` syntax to the actual character, you can specify the `interpretNumericEntities` option as well:

```javascript
var detectedAsIso8859_1 = qs.parse('a=%26%239786%3B', {
charset: 'iso-8859-1',
interpretNumericEntities: true,
})
assert.deepEqual(detectedAsIso8859_1, { a: '☺' })
```

It also works when the charset has been detected in `charsetSentinel` mode.

### Parsing Arrays

**qs** can also parse arrays using a similar `[]` notation:
Expand Down Expand Up @@ -346,6 +326,8 @@ var encodedValues = qs.stringify(
assert.equal(encodedValues, 'a=b&c[0]=d&c[1]=e%3Df&f[0][0]=g&f[1][0]=h')
```

### Encoding

This encoding can also be replaced by a custom encoding method set as `encoder` option:

```javascript
Expand Down Expand Up @@ -373,13 +355,13 @@ var decoded = qs.parse('x=z', {
})
```

You can encode keys and values using different logic by using the type argument provided to the encoder:
You can encode keys and values using different logic by using the `type` argument provided to the encoder:

```javascript
var encoded = qs.stringify(
{ a: { b: 'c' } },
{
encoder: function (str, defaultEncoder, charset, type) {
encoder: function (str, defaultEncoder, type) {
if (type === 'key') {
return // Encoded key
} else if (type === 'value') {
Expand All @@ -390,11 +372,11 @@ var encoded = qs.stringify(
)
```

The type argument is also provided to the decoder:
The `type` argument is also provided to the decoder:

```javascript
var decoded = qs.parse('x=z', {
decoder: function (str, defaultDecoder, charset, type) {
decoder: function (str, defaultDecoder, type) {
if (type === 'key') {
return // Decoded key
} else if (type === 'value') {
Expand Down Expand Up @@ -633,52 +615,6 @@ var nullsSkipped = qs.stringify({ a: 'b', c: null }, { skipNulls: true })
assert.equal(nullsSkipped, 'a=b')
```

If you're communicating with legacy systems, you can switch to `iso-8859-1` using the `charset` option:

```javascript
var iso = qs.stringify({ æ: 'æ' }, { charset: 'iso-8859-1' })
assert.equal(iso, '%E6=%E6')
```

Characters that don't exist in `iso-8859-1` will be converted to numeric entities, similar to what browsers do:

```javascript
var numeric = qs.stringify({ a: '☺' }, { charset: 'iso-8859-1' })
assert.equal(numeric, 'a=%26%239786%3B')
```

You can use the `charsetSentinel` option to announce the character by including an `utf8=✓` parameter with the proper encoding if the checkmark, similar to what Ruby on Rails and others do when submitting forms.

```javascript
var sentinel = qs.stringify({ a: '☺' }, { charsetSentinel: true })
assert.equal(sentinel, 'utf8=%E2%9C%93&a=%E2%98%BA')

var isoSentinel = qs.stringify({ a: 'æ' }, { charsetSentinel: true, charset: 'iso-8859-1' })
assert.equal(isoSentinel, 'utf8=%26%2310003%3B&a=%E6')
```

### Dealing with special character sets

By default the encoding and decoding of characters is done in `utf-8`, and `iso-8859-1` support is also built in via the `charset` parameter.

If you wish to encode querystrings to a different character set (i.e.
[Shift JIS](https://en.wikipedia.org/wiki/Shift_JIS)) you can use the
[`qs-iconv`](https://github.com/martinheidegger/qs-iconv) library:

```javascript
var encoder = require('qs-iconv/encoder')('shift_jis')
var shiftJISEncoded = qs.stringify({ a: 'こんにちは！' }, { encoder: encoder })
assert.equal(shiftJISEncoded, 'a=%82%B1%82%F1%82%C9%82%BF%82%CD%81I')
```

This also works for decoding of query strings:

```javascript
var decoder = require('qs-iconv/decoder')('shift_jis')
var obj = qs.parse('a=%82%B1%82%F1%82%C9%82%BF%82%CD%81I', { decoder: decoder })
assert.deepEqual(obj, { a: 'こんにちは！' })
```

### RFC 3986 and RFC 1738 space encoding

RFC3986 used as default option and encodes ' ' to _%20_ which is backward compatible.
Expand Down
13 changes: 4 additions & 9 deletions lib/index.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
export as namespace QSESM

export type defaultEncoder = (str: any, defaultEncoder?: any, charset?: string) => string
export type defaultDecoder = (str: string, decoder?: any, charset?: string) => string
export type defaultEncoder = (str: any, defaultEncoder?: any) => string
export type defaultDecoder = (str: string, decoder?: any) => string

export type BooleanOptional = boolean | undefined

Expand All @@ -11,7 +11,7 @@ export interface IStringifyBaseOptions {
skipNulls?: boolean | undefined
encode?: boolean | undefined
encoder?:
| ((str: any, defaultEncoder: defaultEncoder, charset: string, type: 'key' | 'value') => string)
| ((str: any, defaultEncoder: defaultEncoder, type: 'key' | 'value') => string)
| undefined
filter?: Array<string | number> | ((prefix: string, value: any) => any) | undefined
arrayFormat?: 'indices' | 'brackets' | 'repeat' | 'comma' | undefined
Expand All @@ -21,8 +21,6 @@ export interface IStringifyBaseOptions {
format?: 'RFC1738' | 'RFC3986' | undefined
encodeValuesOnly?: boolean | undefined
addQueryPrefix?: boolean | undefined
charset?: 'utf-8' | 'iso-8859-1' | undefined
charsetSentinel?: boolean | undefined
}

export type IStringifyDynamicOptions<AllowDots extends BooleanOptional> = AllowDots extends true
Expand All @@ -37,7 +35,7 @@ export interface IParseBaseOptions {
delimiter?: string | RegExp | undefined
depth?: number | false | undefined
decoder?:
| ((str: string, defaultDecoder: defaultDecoder, charset: string, type: 'key' | 'value') => any)
| ((str: string, defaultDecoder: defaultDecoder, type: 'key' | 'value') => any)
| undefined
arrayLimit?: number | undefined
parseArrays?: boolean | undefined
Expand All @@ -47,9 +45,6 @@ export interface IParseBaseOptions {
parameterLimit?: number | undefined
strictNullHandling?: boolean | undefined
ignoreQueryPrefix?: boolean | undefined
charset?: 'utf-8' | 'iso-8859-1' | undefined
charsetSentinel?: boolean | undefined
interpretNumericEntities?: boolean | undefined
allowEmptyArrays?: boolean | undefined
duplicates?: 'combine' | 'first' | 'last' | undefined
}
Expand Down
64 changes: 3 additions & 61 deletions lib/parse.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,28 +11,19 @@ const defaults = {
allowPrototypes: false,
allowSparse: false,
arrayLimit: 20,
charset: 'utf-8',
charsetSentinel: false,
comma: false,
decodeDotInKeys: false,
decoder: utils.decode,
delimiter: '&',
depth: 5,
duplicates: 'combine',
ignoreQueryPrefix: false,
interpretNumericEntities: false,
parameterLimit: 1000,
parseArrays: true,
plainObjects: false,
strictNullHandling: false,
}

const interpretNumericEntities = function (str) {
return str.replace(/&#(\d+);/g, function ($0, numberStr) {
return String.fromCharCode(parseInt(numberStr, 10))
})
}

const parseArrayValue = function (val, options) {
if (val && typeof val === 'string' && options.comma && val.indexOf(',') > -1) {
return val.split(',')
Expand All @@ -41,64 +32,31 @@ const parseArrayValue = function (val, options) {
return val
}

// This is what browsers will submit when the ✓ character occurs in an
// application/x-www-form-urlencoded body and the encoding of the page containing
// the form is iso-8859-1, or when the submitted form has an accept-charset
// attribute of iso-8859-1. Presumably also with other charsets that do not contain
// the ✓ character, such as us-ascii.
const isoSentinel = 'utf8=%26%2310003%3B' // encodeURIComponent('&#10003;')

// These are the percent-encoded utf-8 octets representing a checkmark, indicating that the request actually is utf-8 encoded.
const charsetSentinel = 'utf8=%E2%9C%93' // encodeURIComponent('✓')

const parseValues = function parseQueryStringValues(str, options) {
const obj = { __proto__: null }

const cleanStr = options.ignoreQueryPrefix ? str.replace(/^\?/, '') : str
const limit = options.parameterLimit === Infinity ? undefined : options.parameterLimit
const parts = cleanStr.split(options.delimiter, limit)
let skipIndex = -1 // Keep track of where the utf8 sentinel was found
let i

let charset = options.charset
if (options.charsetSentinel) {
for (i = 0; i < parts.length; ++i) {
if (parts[i].indexOf('utf8=') === 0) {
if (parts[i] === charsetSentinel) {
charset = 'utf-8'
} else if (parts[i] === isoSentinel) {
charset = 'iso-8859-1'
}
skipIndex = i
i = parts.length // The eslint settings do not allow break;
}
}
}

for (i = 0; i < parts.length; ++i) {
if (i === skipIndex) {
continue
}
const part = parts[i]

const bracketEqualsPos = part.indexOf(']=')
const pos = bracketEqualsPos === -1 ? part.indexOf('=') : bracketEqualsPos + 1

let key, val
if (pos === -1) {
key = options.decoder(part, defaults.decoder, charset, 'key')
key = options.decoder(part, defaults.decoder, 'key')
val = options.strictNullHandling ? null : ''
} else {
key = options.decoder(part.slice(0, pos), defaults.decoder, charset, 'key')
key = options.decoder(part.slice(0, pos), defaults.decoder, 'key')
val = utils.maybeMap(parseArrayValue(part.slice(pos + 1), options), function (encodedVal) {
return options.decoder(encodedVal, defaults.decoder, charset, 'value')
return options.decoder(encodedVal, defaults.decoder, 'value')
})
}

if (val && options.interpretNumericEntities && charset === 'iso-8859-1') {
val = interpretNumericEntities(val)
}

if (part.indexOf('[]=') > -1) {
val = isArray(val) ? [val] : val
}
Expand Down Expand Up @@ -235,15 +193,6 @@ const normalizeParseOptions = function normalizeParseOptions(opts) {
throw new TypeError('Decoder has to be a function.')
}

if (
typeof opts.charset !== 'undefined' &&
opts.charset !== 'utf-8' &&
opts.charset !== 'iso-8859-1'
) {
throw new TypeError('The charset option must be either utf-8, iso-8859-1, or undefined')
}
const charset = typeof opts.charset === 'undefined' ? defaults.charset : opts.charset

const duplicates = typeof opts.duplicates === 'undefined' ? defaults.duplicates : opts.duplicates

if (duplicates !== 'combine' && duplicates !== 'first' && duplicates !== 'last') {
Expand All @@ -267,9 +216,6 @@ const normalizeParseOptions = function normalizeParseOptions(opts) {
typeof opts.allowPrototypes === 'boolean' ? opts.allowPrototypes : defaults.allowPrototypes,
allowSparse: typeof opts.allowSparse === 'boolean' ? opts.allowSparse : defaults.allowSparse,
arrayLimit: typeof opts.arrayLimit === 'number' ? opts.arrayLimit : defaults.arrayLimit,
charset: charset,
charsetSentinel:
typeof opts.charsetSentinel === 'boolean' ? opts.charsetSentinel : defaults.charsetSentinel,
comma: typeof opts.comma === 'boolean' ? opts.comma : defaults.comma,
decodeDotInKeys:
typeof opts.decodeDotInKeys === 'boolean' ? opts.decodeDotInKeys : defaults.decodeDotInKeys,
Expand All @@ -282,10 +228,6 @@ const normalizeParseOptions = function normalizeParseOptions(opts) {
depth: typeof opts.depth === 'number' || opts.depth === false ? +opts.depth : defaults.depth,
duplicates: duplicates,
ignoreQueryPrefix: opts.ignoreQueryPrefix === true,
interpretNumericEntities:
typeof opts.interpretNumericEntities === 'boolean'
? opts.interpretNumericEntities
: defaults.interpretNumericEntities,
parameterLimit:
typeof opts.parameterLimit === 'number' ? opts.parameterLimit : defaults.parameterLimit,
parseArrays: opts.parseArrays !== false,
Expand Down
Loading
Loading