Base64 and URL Encoding Explained, and Why btoa() Fails on Arabic

Type btoa('مرحبا') into a browser console and you get an InvalidCharacterError. Try btoa('café') and it works, but a server decoding the result as UTF-8 gets garbage. Both encodings behind ToolOnTap’s Base64 and URL tools look trivial, and both have a trap that only shows up once your text isn’t plain English. Arabic text walks straight into both traps, so I built the tools around them.

Text is bytes first

Base64 and percent-encoding both work on bytes. Text has to become bytes before either can touch it, and the rule for that step decides whether the result round-trips.

UTF-8 is the rule the web uses. Plain English letters take 1 byte each, accented Latin letters and Arabic letters take 2, and most emoji take 4. “مرحبا” is five letters and 10 bytes. “café ☕” is six characters and 9 bytes.

btoa() predates all of that. It takes a string where every character must have a code below 256 and treats each character as one byte. Arabic letters are far above 255, so it throws. é is 233, so it passes, but it’s written as the single byte E9, the Latin-1 form, instead of the two UTF-8 bytes C3 A9. The output, Y2Fm6Q==, decodes to broken text everywhere that expects UTF-8. The correct UTF-8 Base64 for “café” is Y2Fmw6k=.

So both tools always go through TextEncoder first, which produces UTF-8 bytes, and only then encode:

export function bytesToBase64(bytes: Uint8Array, urlSafe = false): string {
  let bin = ''
  const CHUNK = 0x8000  // String.fromCharCode(...huge) would overflow the call stack
  for (let i = 0; i < bytes.length; i += CHUNK) bin += String.fromCharCode(...bytes.subarray(i, i + CHUNK))
  const b64 = btoa(bin)
  return urlSafe ? b64.replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '') : b64
}

Each byte becomes one character with a code from 0 to 255, which is exactly what btoa() accepts. The chunking matters for files: spreading a few megabytes of bytes into one String.fromCharCode(...) call overflows the call stack, so it goes 32,768 bytes at a time. The tests run a 1,000,000-byte array through it and compare the result with Node’s own Base64.

How Base64 works

Base64 takes 3 bytes (24 bits), cuts them into four 6-bit groups, and writes each group as one of 64 characters: A–Z, a–z, 0–9, + and /. That’s why encoded data is a third larger than the original.

When the input isn’t a multiple of 3 bytes, the last group is padded with =. One leftover byte gives two characters and ==, two leftover bytes give three characters and =. “hello” is 5 bytes, so it ends with one =: aGVsbG8=.

That rule also gives a cheap way to catch a damaged string. Without its padding, a valid Base64 string can never have a length that leaves 1 when divided by 4, because a lone last character holds only 6 bits, too few for a byte. The decoder checks exactly that and says “one character too long or too short” instead of guessing.

Base64URL: same idea, URL-safe alphabet

+ and / mean something in URLs, and = does too in query strings. Base64URL (RFC 4648, section 5) swaps + for - and / for _, and usually drops the padding. The same three bytes FB FF BF are +/+/ in standard Base64 and -_-_ in Base64URL. JWTs use it, and so do many APIs that put binary IDs in links.

The “URL-safe (Base64URL, no padding)” option does that swap when encoding. When decoding, you don’t choose: the decoder accepts both alphabets, with or without padding, and ignores line breaks, so Base64 copied from an email or a wrapped config value works as it is.

What people paste is often messier than a clean string, so the decoder also:

• strips a data:…;base64, prefix, so a data URI from HTML or CSS decodes directly;
• points at the first bad character with its position, such as ““$” isn’t a Base64 character (position 4)”;
• tries to read the bytes as UTF-8 in strict mode, and if they aren’t text, looks at the first bytes to recognize a PNG, JPEG, GIF, PDF, ZIP or WebP file and offers it as a download.

Percent-encoding has a different job

URL encoding solves another problem. A URL can contain only a limited set of characters, and some of those have a structural meaning: / separates path parts, ? starts the query, & and = separate parameters, # starts the fragment. Any other byte is written as % followed by two hex digits. A space becomes %20.

Non-English text goes through UTF-8 here too. “مرحبا” becomes %D9%85%D8%B1%D8%AD%D8%A8%D8%A7, two escapes per letter because each letter is 2 bytes. é becomes %C3%A9.

The real decision is what you’re encoding, and the tool asks it directly:

Encoding a value as “A full URL” leaves its & alone, and the server then sees two parameters where you meant one. Encoding a full URL as “A value” turns https:// into https%3A%2F%2F. The test file has one line for each: a b&c=d/é as a value is a%20b%26c%3Dd%2F%C3%A9, and a full URL keeps its ?, = and &.

When + means a space

HTML forms send data as application/x-www-form-urlencoded, where a space is written as +. In a URL path, a + is just a plus sign. Both conventions are in daily use, so neither can be the default, and the tool makes it an option: “Spaces as +” when encoding and “Read + as a space” when decoding, both marked “(form data)”.

The encoding side is a single replacement after encodeURIComponent. That’s safe because encodeURIComponent has already turned every real plus sign into %2B, so the only + signs in the output are the former spaces. a+b c becomes a%2Bb+c, and decoding it with the option ticked gives a+b c back.

The decoder is also strict where browsers’ own function fails with a bare URIError. A literal % that isn’t followed by two hex digits, as in “100% sure”, is reported with its position and the fix (write it as %25). Escapes that don’t form valid UTF-8, such as a lone %C3, get their own message, because that usually means the text was cut in the middle of a character or encoded with another character set.

Reading long links

Decode a full URL and the tool also splits it into protocol, host, path, fragment and query parameters, with each value decoded. It uses the browser’s URL parser for that, so it splits the link the same way the browser does. Repeated parameters, such as tag=a&tag=b, stay as separate rows instead of being merged. That’s the view I wanted for tracking links and redirects, where the real destination is usually a parameter several escapes deep.

Try it

Paste text into the Base64 encoder and decoder or the URL encoder and decoder. Both convert as you type, and the “Decode this” button sends the result back the other way, so you can check that it round-trips. For images in HTML or CSS, Image to Base64 gives you a ready data URI. The rest are on the developer tools page.