URI

is a URI datatype and parser for Carp.

Installation

(load "git@github.com:carpentry-org/uri@0.6.0")

Usage

To get started, you’ll most probably want to parse a URI from a string. To do so, you use URI.parse. This function will return a URI datatype for you to work with, or an error type if the URI string was invalid.

If you have a URI value the simplest operation is probably converting the URI back to a string using the str interface, which should be idempotent—i.e. you’ll get the original URI back. Components are stored exactly as they were written, so path keeps its leading /.

You can also ask the URI for its properties, like the scheme, the port, or the URI parameters.

A more complete documentation can be found under https://veitheller.de/uri/!

Acknowledgements

This datatype and parser was heavily inspired by the one in the Crystal standard library. I cannot thank the people who worked on it enough; they saved me from going through a lot of pain and suffering!

=

defn

(Fn [(Ref URI a), (Ref URI b)] Bool) 

                        (= u1 u2)

is defined as the equality of all members of URIs u1 and u2.

absolute?

defn

(Fn [(Ref URI a)] Bool) 

                        (absolute? u)

checks whether a URI u is absolute.

copy

instantiate

(Fn [(Ref URI a)] URI)

copies a URI.

default-port

defn

(Fn [(Maybe String)] Int) 

                        (default-port s)

gets the default port for the scheme of the URI u. Returns 0 if it’s unknown.

default-port?

defn

(Fn [(Ref URI a)] Bool) 

                        (default-port? u)

checks whether the port of the URI u is the default port for its scheme.

default-ports

def

(Map String Int)

is a map of all the services that have default ports—that we know of—and their port values.

delete

instantiate

(Fn [URI] ())

deletes a URI. Should usually not be called manually.

escape

defn

(Fn [(Ref String a)] String) 

                        (escape s)

URI escapes the string s, percent-encoding every byte that is not an unreserved ASCII character. Multi-byte UTF-8 characters are encoded one byte at a time, so the output is standard, interoperable percent-encoding.

fragment

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the fragment property of a URI.

full-path

defn

(Fn [(Ref URI a)] String) 

                        (full-path u)

returns the full path of a URI u, i.e. the origin-form request target: the path followed by ? and the query when one is present.

The result always begins with /; a URI with no path (or an empty path) yields /.

(def uri (URI.parse "http://foo.com/posts?id=30&limit=5#time=1305298413"))
(URI.full-path &uri) ; => /posts?id=30&limit=5

host

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the host property of a URI.

hostname

defn

(Fn [(Ref URI a)] (Maybe String)) 

                        (hostname u)

returns the host part of a URI u, unwrapping brackets for IPv6 addresses.

(URI.hostname &(URI.parse "http://[::1]/bar"))   ; => ::1
(URI.hostname &(URI.parse "http://foo.com/bar")) ; => foo.com

init

instantiate

(Fn [(Maybe String), (Maybe String), (Maybe Int), (Maybe String), (Maybe String), (Maybe String), (Maybe String), (Maybe String), (Maybe String)] URI)

creates a URI.

normalize

defn

(Fn [(Ref URI StaticLifetime)] URI) 

                        (normalize u)

normalizes a URI u following RFC 3986 section 6.2, returning an equivalent URI in a canonical form. This makes two URIs that denote the same resource compare equal under URI.=.

The following transformations are applied:

  • Case normalization (§6.2.2.1): the scheme and host are lower-cased, and the hexadecimal digits of percent-encoded triplets are upper-cased.
  • Percent-encoding normalization (§6.2.2.2): percent-encoded octets that correspond to unreserved characters (ALPHA / DIGIT / - / . / _ / ~) are decoded to their literal form.
  • Path segment normalization (§6.2.2.3): . and .. segments are removed from the path.
  • Default-port removal (§6.2.3): a port equal to the scheme’s default is dropped.

Case-sensitive components (userinfo, path, query, and fragment) keep their case; only their percent-encoding is normalized.

Note: because . is unreserved, a percent-encoded dot (%2E) is decoded (§6.2.2.2) and then resolved by dot-segment removal (§6.2.2.3), so /a/%2E%2E/b normalizes to /b. An encoded .. therefore collapses just like a literal one, which callers using normalize for path-prefix security checks should account for.

(URI.str &(URI.normalize &(Result.unsafe-from-success
  (URI.parse "HTTP://User@Example.COM:80/%7Ejoe/./index.html"))))
; => “http://User@example.com/~joe/index.html”

opaque

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the opaque property of a URI.

parse

defn

(Fn [(Ref String a)] (Result URI String)) 

                        (parse s)

Parses a URI from a string s.

password

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the password property of a URI.

path

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the path property of a URI.

port

instantiate

(Fn [(Ref URI a)] (Ref (Maybe Int) a))

gets the port property of a URI.

prn

instantiate

(Fn [(Ref URI a)] String)

converts a URI to a string.

query

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the query property of a URI.

query-map

defn

(Fn [(Ref URI a)] (Result (Map String String) String)) 

                        (query-map u)

parses the querystring as a Map and returns it, percent-decoding keys and values.

A parameter without a value, such as flag in ?flag, is stored with an empty string as its value. Empty segments, such as a trailing or doubled &, are skipped. The result is wrapped in a Result for API compatibility.

query-map-from-str

defn

(Fn [(Ref String a)] (Result (Map String String) String)) 

                        (query-map-from-str s)

parses the querystring as a Map and returns it, percent-decoding keys and values.

A parameter without a value, such as flag in ?flag, is stored with an empty string as its value. Empty segments, such as a trailing or doubled &, are skipped. The result is wrapped in a Result for API compatibility.

query-multimap

defn

(Fn [(Ref URI a)] (Result (Map String (Array String)) String)) 

                        (query-multimap u)

parses the querystring, collecting every value for a key into an Array in encounter order, and returns the result, percent-decoding keys and values.

This mirrors query-map but preserves repeated keys; see query-multimap-from-str for the exact edge-case semantics.

query-multimap-from-str

defn

(Fn [(Ref String a)] (Result (Map String (Array String)) String)) 

                        (query-multimap-from-str s)

parses the querystring, collecting every value for a key into an Array in encounter order, and returns the result, percent-decoding keys and values.

Unlike query-map-from-str, repeated keys such as a in a=1&a=2 are all kept instead of overwritten. A parameter without a value, such as flag in ?flag, contributes a single empty-string value. Empty segments, such as a trailing or doubled &, are skipped. The result is wrapped in a Result for API compatibility.

query-string-from-map

defn

(Fn [(Ref (Map String String) a)] String) 

                        (query-string-from-map m)

converts a Map of query parameters back into a querystring, percent-encoding keys and values.

This is the inverse of query-map-from-str.

query-values

defn

(Fn [(Ref URI a), (Ref String b)] (Array String)) 

                        (query-values u k)

returns every value supplied for the query parameter k in URI u, in encounter order and percent-decoded. A key that is absent yields an empty Array.

relative?

defn

(Fn [(Ref URI a)] Bool) 

                        (relative? u)

checks whether a URI u is relative.

remove-dot-segments

defn

(Fn [(Ref String a)] String) 

                        (remove-dot-segments path)

normalizes a path by resolving . and .. segments following RFC 3986 section 5.2.4.

Runs in a single linear pass: a byte cursor walks the input once and output segments are collected on a stack, so an n-segment path costs O(n) instead of the quadratic slicing a literal transcription of the RFC would incur.

resolve

defn

(Fn [(Ref URI a), (Ref String b)] (Result URI String)) 

                        (resolve base ref-str)

resolves a URI reference string ref-str against a base URI base, following RFC 3986 section 5. Returns the resolved absolute URI, or an error if the reference string cannot be parsed.

(def base (Result.unsafe-from-success (URI.parse "http://a/b/c/d?q")))
(URI.resolve &base "../g")   ; => http://a/b/g
(URI.resolve &base "/g")     ; => http://a/g
(URI.resolve &base "?y")     ; => http://a/b/c/d?y

scheme

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the scheme property of a URI.

set-fragment

instantiate

(Fn [URI, (Maybe String)] URI)

sets the fragment property of a URI.

set-fragment!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the fragment property of a URI in place.

set-host

instantiate

(Fn [URI, (Maybe String)] URI)

sets the host property of a URI.

set-host!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the host property of a URI in place.

set-opaque

instantiate

(Fn [URI, (Maybe String)] URI)

sets the opaque property of a URI.

set-opaque!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the opaque property of a URI in place.

set-password

instantiate

(Fn [URI, (Maybe String)] URI)

sets the password property of a URI.

set-password!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the password property of a URI in place.

set-path

instantiate

(Fn [URI, (Maybe String)] URI)

sets the path property of a URI.

set-path!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the path property of a URI in place.

set-port

instantiate

(Fn [URI, (Maybe Int)] URI)

sets the port property of a URI.

set-port!

instantiate

(Fn [(Ref URI a), (Maybe Int)] ())

sets the port property of a URI in place.

set-query

instantiate

(Fn [URI, (Maybe String)] URI)

sets the query property of a URI.

set-query!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the query property of a URI in place.

set-scheme

instantiate

(Fn [URI, (Maybe String)] URI)

sets the scheme property of a URI.

set-scheme!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the scheme property of a URI in place.

set-user

instantiate

(Fn [URI, (Maybe String)] URI)

sets the user property of a URI.

set-user!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the user property of a URI in place.

str

defn

(Fn [(Ref URI a)] String) 

                        (str u)

prints the URL u as idempotently as possible, i.e. as the parsed string.

(def uri (URI.parse "http://admin:password@foo.com"))
(URI.str &uri) ; => "http://admin:password@foo.com"

unescape

defn

(Fn [(Ref String a)] String) 

                        (unescape s)

URI unescapes the string s, decoding each percent-triplet into a single byte and copying all other bytes through verbatim. UTF-8 characters that were percent-encoded byte by byte are therefore reassembled correctly.

update-fragment

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the fragment property of a URI using a function f.

update-host

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the host property of a URI using a function f.

update-opaque

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the opaque property of a URI using a function f.

update-password

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the password property of a URI using a function f.

update-path

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the path property of a URI using a function f.

update-port

instantiate

(Fn [URI, (Ref (Fn [(Maybe Int)] (Maybe Int) a) b)] URI)

updates the port property of a URI using a function f.

update-query

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the query property of a URI using a function f.

update-scheme

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the scheme property of a URI using a function f.

update-user

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the user property of a URI using a function f.

user

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the user property of a URI.

userinfo

defn

(Fn [(Ref URI a)] String) 

                        (userinfo u)

returns the user-information component for a URI u, which contains the provided username and password.

(def uri (URI.parse "http://admin:password@foo.com"))
(URI.userinfo &uri) ; => admin:password

zero

defn

(Fn [] URI) 

                        (zero)

Creates an empty URI, i.e. one that has only Nothing in it.