URL Encoding Explained: Percent-Encoding, Query Strings and Common Bugs

A user searches your site for fish & chips. The results page comes back showing matches for fish. Nobody typed anything wrong, nothing threw an error, and the bug will survive several rounds of code review because the code that caused it looks completely reasonable.

That is URL encoding. It is a small topic that produces a disproportionate number of subtle bugs, and nearly all of them come from two or three specific misunderstandings.

Why URLs have a restricted character set

A URL is not free-form text. It is a structured string where certain characters are punctuation telling a parser where each part begins and ends:

https://example.com/search?q=shoes&page=2#results
        └── host ──┘└path┘ └── query ──┘ └frag┘

The ? starts the query. & separates parameters. = splits a key from its value. / separates path segments. # starts the fragment.

So what happens when a value needs to contain one of those characters? If a search term contains &, the parser cannot tell that ampersand apart from the one separating parameters. It splits there, and half the search term becomes a parameter nobody asked for.

Percent-encoding is the escape hatch.

Reserved vs unreserved characters

The URI specification sorts characters into groups. Unreserved characters are always safe anywhere and never need encoding:

A–Z  a–z  0–9  -  .  _  ~

Reserved characters carry structural meaning and must be encoded when they appear inside a value rather than as syntax:

:  /  ?  #  [  ]  @                 (delimiters between major parts)
!  $  &  '  (  )  *  +  ,  ;  =     (delimiters within parts)

Everything else — spaces, accented letters, emoji, most punctuation — has to be encoded to appear in a URL at all.

How percent-encoding works

The mechanism is simple: replace the character with a % followed by the two hex digits of its byte value.

• Space is byte 0x20 → %20
• / is 0x2F → %2F
• & is 0x26 → %26
• % itself is 0x25 → %25

The word byte in that sentence is the part people skip, and it matters enormously.

Non-English text and UTF-8

Percent-encoding encodes bytes, not characters. Anything outside ASCII has to become bytes first, and which bytes you get depends on the character encoding. Modern practice is UTF-8 everywhere.

Take é (U+00E9). In UTF-8 that is two bytes, C3 A9, so it encodes as %C3%A9. Under the older Latin-1 encoding it would be a single byte, %E9. Same character, different output — and a server expecting one while receiving the other produces classic mojibake, with café arriving as café.

The rule: encode as UTF-8, decode as UTF-8, on both ends. If accented characters or emoji come through mangled, an encoding mismatch is the first thing to check.

encodeURI vs encodeURIComponent

JavaScript gives you two functions, and picking the wrong one causes the bug at the top of this article.

encodeURI is for encoding a whole URL. It deliberately leaves reserved characters alone, because in a complete URL those characters are doing their structural job. It will not encode : / ? # & = + $ , ; @.

encodeURIComponent is for encoding a single value being inserted into a URL. It encodes reserved characters too, because inside a value they are data, not syntax.

Here is the failure, concretely:

const term = "fish & chips";

// WRONG — encodeURI leaves the ampersand intact
"/search?q=" + encodeURI(term);
// "/search?q=fish%20&%20chips"
// The server sees TWO parameters: q="fish " and "%20chips"=""

// RIGHT
"/search?q=" + encodeURIComponent(term);
// "/search?q=fish%20%26%20chips"
// The server sees one parameter: q="fish & chips"

The same trap appears with slashes. A value like docs/intro passed through encodeURI keeps its / and gets read as an extra path segment; encodeURIComponent turns it into docs%2Fintro, which stays a single value.

The practical rule: use encodeURIComponent on individual values, and encodeURI almost never. If you are reaching for encodeURI, you are usually building a URL by string concatenation when you should be building it structurally:

const url = new URL("https://example.com/search");
url.searchParams.set("q", "fish & chips");
url.toString(); // https://example.com/search?q=fish+%26+chips

URL and URLSearchParams encode each part correctly by construction, which removes the whole category of mistake.

One footnote: neither function encodes ! ' ( ) *, which the spec lists as reserved. It rarely matters, but against a strict server-side parser it can — “encodeURIComponent output” and “fully spec-compliant encoding” are not quite the same thing.

The + versus %20 confusion

Spaces have two encodings in the wild, from two different specifications.

• %20 is standard percent-encoding, valid anywhere in a URL.
• + means a space in application/x-www-form-urlencoded — the format HTML forms use, which by convention also governs query strings.

So ?q=fish+and+chips and ?q=fish%20and%20chips are usually treated identically by a web framework parsing a query string. But run ?q=fish+and+chips through a generic percent-decoder and you get the literal string fish+and+chips, plus signs and all.

The trap this creates: a literal plus sign in a value must be encoded as %2B, or something downstream will read it as a space.

tel=%2B44%207700%20900123      correct  → +44 7700 900123
tel=+44%207700%20900123        broken   → " 44 7700 900123"

This bites hardest with phone numbers and tagged email addresses like [email protected] — sign-up forms and unsubscribe links that mishandle it are a genuinely common bug. Notice in the URLSearchParams example above that JavaScript emits + for spaces there, while encodeURIComponent emits %20. Both are correct in their own context; know which one you are producing.

Double-encoding bugs

Double encoding happens when an already-encoded string gets encoded again. Because % itself encodes to %25, the signature is unmistakable once you know it:

Original:        hello world
Encoded once:    hello%20world
Encoded twice:   hello%2520world

The visible symptom is a literal %20 in your page text, a search result for a term containing %2F, or a redirect landing on a 404 with an odd-looking path.

How to spot it: search the URL for %25. In almost every case, %25 followed by two hex digits means something was encoded one time too many.

The usual causes are encoding a value then encoding it again on the way out, passing a full URL as a ?redirect= parameter and encoding at two layers, or adding a manual encodeURIComponent on top of an HTTP client that already encodes for you.

The fix is not to sprinkle in a decodeURIComponent. It is to work out which layer owns the encoding and delete the duplicate. Keep values decoded in memory, and encode exactly once, when you build the URL. To check what you actually have, paste the string into the URL Encoder / Decoder and decode it — if one pass leaves you with a still-encoded string, you found your double encoding.

Encode values, not whole URLs

Almost every bug above comes from treating a URL as one long string. The reliable habit is to treat it as a structure: start from a base URL, add path segments and parameters individually, and encode each value as you add it — never the assembled result.

The same discipline in Python:

from urllib.parse import urlencode, quote

quote("fish & chips")                    # 'fish%20%26%20chips'
urlencode({"q": "fish & chips", "p": 2}) # 'q=fish+%26+chips&p=2'

Note that quote produces %20 while urlencode produces +, mirroring the JavaScript split exactly — one is percent-encoding, the other is form encoding.

A related point: percent-encoding makes text safe inside a URL. To carry binary data instead, that is a different problem — see Base64, and note that standard Base64’s own + and / are URL-unsafe, which is why a URL-safe variant exists. And if you are generating readable URL paths from titles, strip the special characters at the source with the Slug Generator rather than encoding them.

Quick answers

What is %20? A space. 20 is the hexadecimal value of the space character’s byte.

encodeURI or encodeURIComponent? encodeURIComponent for individual values, which is nearly always what you want. encodeURI only for a complete URL you need to make safe without breaking its structure.

Why does + sometimes mean space? It comes from HTML form encoding, which query strings inherited. In the path portion of a URL, + is a literal plus.

How do I spot double encoding? Look for %25 in the URL. That is an encoded %, which almost always means the string went through an encoder twice.

Do I need to encode a whole URL before sending it? No — encode the individual values going into it. Encoding the whole thing escapes the punctuation that makes it a URL.

The takeaway

Percent-encoding exists so data can sit safely inside a string whose punctuation carries meaning. Encode values rather than URLs, use encodeURIComponent or a real URL builder rather than encodeURI, keep everything UTF-8, watch for %25 when things look wrong, and remember that + and %20 come from two different specifications that happen to share a query string.