URI

is a URI datatype and parser for Carp.

Installation

(load "git@github.com:carpentry-org/uri@0.4.0")

Usage

To get started, you’ll most probably want to parse a URI from a string. To do so, you use URI.parse. This function will return a URI datatype for you to work with, or an error type if the URI string was invalid.

If you have a a URI value the simplest operation is probably converting the URI back to a string using the str interface, which should be idempotent—i.e. you’ll get the original URI back.

You can also ask the URI for its properties, like the scheme, the port, or the URI parameters.

A more complete documentation can be found under https://veitheller.de/uri/!

Acknowledgements

This datatype and parser was heavily inspired by the one in the Crystal standard library. I cannot thank the people who worked on it enough; they saved me from going through a lot of pain and suffering!

=

defn

(Fn [(Ref URI a), (Ref URI b)] Bool) 

                        (= u1 u2)

is defined as the equality of all members of URIs u1 and u2.

absolute?

defn

(Fn [(Ref URI a)] Bool) 

                        (absolute? u)

checks whether a URI u is absolute.

copy

instantiate

(Fn [(Ref URI a)] URI)

copies a URI.

default-port

defn

(Fn [(Maybe String)] Int) 

                        (default-port s)

gets the default port for the scheme of the URI u. Returns 0 if it’s unknown.

default-port?

defn

(Fn [(Ref URI a)] Bool) 

                        (default-port? u)

checks whether the port of the URI u is the default port for its scheme.

default-ports

def

(Map String Int)

is a map of all the services that have default ports—that we know of—and their port values.

delete

instantiate

(Fn [URI] ())

deletes a URI. Should usually not be called manually.

escape

defn

(Fn [(Ref String a)] String) 

                        (escape s)

URI escapes the string s, percent-encoding every byte that is not an unreserved ASCII character. Multi-byte UTF-8 characters are encoded one byte at a time, so the output is standard, interoperable percent-encoding.

fragment

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the fragment property of a URI.

full-path

defn

(Fn [(Ref URI a)] String) 

                        (full-path u)

returns the full path of a URI u, i.e. the origin-form request target: the path followed by ? and the query when one is present.

The result always begins with /; a URI with no path (or an empty path) yields /.

(def uri (URI.parse "http://foo.com/posts?id=30&limit=5#time=1305298413"))
(URI.full-path &uri) ; => /posts?id=30&limit=5

host

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the host property of a URI.

hostname

defn

(Fn [(Ref URI a)] (Maybe String)) 

                        (hostname u)

returns the host part of a URI u, unwrapping brackets for IPv6 addresses.

(URI.hostname &(URI.parse "http://[::1]/bar")) ; => ::1
(URI.hostname &(URI.parse "http://[::1]/bar")) ; => [::1]

init

instantiate

(Fn [(Maybe String), (Maybe String), (Maybe Int), (Maybe String), (Maybe String), (Maybe String), (Maybe String), (Maybe String), (Maybe String)] URI)

creates a URI.

normalize

defn

(Fn [(Ref URI StaticLifetime)] URI) 

                        (normalize u)

normalizes a URI u following RFC 3986 section 6.2, returning an equivalent URI in a canonical form. This makes two URIs that denote the same resource compare equal under URI.=.

The following transformations are applied:

  • Case normalization (§6.2.2.1): the scheme and host are lower-cased, and the hexadecimal digits of percent-encoded triplets are upper-cased.
  • Percent-encoding normalization (§6.2.2.2): percent-encoded octets that correspond to unreserved characters (ALPHA / DIGIT / - / . / _ / ~) are decoded to their literal form.
  • Path segment normalization (§6.2.2.3): . and .. segments are removed from the path.
  • Default-port removal (§6.2.3): a port equal to the scheme’s default is dropped.

Case-sensitive components (userinfo, path, query, and fragment) keep their case; only their percent-encoding is normalized.

Note: because . is unreserved, a percent-encoded dot (%2E) is decoded (§6.2.2.2) and then resolved by dot-segment removal (§6.2.2.3), so /a/%2E%2E/b normalizes to /b. An encoded .. therefore collapses just like a literal one, which callers using normalize for path-prefix security checks should account for.

(URI.str &(URI.normalize &(Result.unsafe-from-success
  (URI.parse "HTTP://User@Example.COM:80/%7Ejoe/./index.html"))))
; => “http://User@example.com/~joe/index.html”

opaque

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the opaque property of a URI.

parse

defn

(Fn [(Ref String a)] (Result URI String)) 

                        (parse s)

Parses a URI from a string s.

password

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the password property of a URI.

path

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the path property of a URI.

port

instantiate

(Fn [(Ref URI a)] (Ref (Maybe Int) a))

gets the port property of a URI.

prn

instantiate

(Fn [(Ref URI a)] String)

converts a URI to a string.

query

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the query property of a URI.

query-map

defn

(Fn [(Ref URI a)] (Result (Map String String) String)) 

                        (query-map u)

parses the querystring as a Map and returns it, percent-decoding keys and values.

A parameter without a value, such as flag in ?flag, is stored with an empty string as its value. Empty segments, such as a trailing or doubled &, are skipped. The result is wrapped in a Result for API compatibility.

query-map-from-str

defn

(Fn [(Ref String a)] (Result (Map String String) String)) 

                        (query-map-from-str s)

parses the querystring as a Map and returns it, percent-decoding keys and values.

A parameter without a value, such as flag in ?flag, is stored with an empty string as its value. Empty segments, such as a trailing or doubled &, are skipped. The result is wrapped in a Result for API compatibility.

query-string-from-map

defn

(Fn [(Ref (Map String String) a)] String) 

                        (query-string-from-map m)

converts a Map of query parameters back into a querystring, percent-encoding keys and values.

This is the inverse of query-map-from-str.

relative?

defn

(Fn [(Ref URI a)] Bool) 

                        (relative? u)

checks whether a URI u is relative.

remove-dot-segments

defn

(Fn [(Ref String a)] String) 

                        (remove-dot-segments path)

normalizes a path by resolving . and .. segments following RFC 3986 section 5.2.4.

resolve

defn

(Fn [(Ref URI a), (Ref String b)] (Result URI String)) 

                        (resolve base ref-str)

resolves a URI reference string ref-str against a base URI base, following RFC 3986 section 5. Returns the resolved absolute URI, or an error if the reference string cannot be parsed.

(def base (Result.unsafe-from-success (URI.parse "http://a/b/c/d?q")))
(URI.resolve &base "../g")   ; => http://a/b/g
(URI.resolve &base "/g")     ; => http://a/g
(URI.resolve &base "?y")     ; => http://a/b/c/d?y

scheme

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the scheme property of a URI.

set-fragment

instantiate

(Fn [URI, (Maybe String)] URI)

sets the fragment property of a URI.

set-fragment!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the fragment property of a URI in place.

set-host

instantiate

(Fn [URI, (Maybe String)] URI)

sets the host property of a URI.

set-host!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the host property of a URI in place.

set-opaque

instantiate

(Fn [URI, (Maybe String)] URI)

sets the opaque property of a URI.

set-opaque!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the opaque property of a URI in place.

set-password

instantiate

(Fn [URI, (Maybe String)] URI)

sets the password property of a URI.

set-password!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the password property of a URI in place.

set-path

instantiate

(Fn [URI, (Maybe String)] URI)

sets the path property of a URI.

set-path!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the path property of a URI in place.

set-port

instantiate

(Fn [URI, (Maybe Int)] URI)

sets the port property of a URI.

set-port!

instantiate

(Fn [(Ref URI a), (Maybe Int)] ())

sets the port property of a URI in place.

set-query

instantiate

(Fn [URI, (Maybe String)] URI)

sets the query property of a URI.

set-query!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the query property of a URI in place.

set-scheme

instantiate

(Fn [URI, (Maybe String)] URI)

sets the scheme property of a URI.

set-scheme!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the scheme property of a URI in place.

set-user

instantiate

(Fn [URI, (Maybe String)] URI)

sets the user property of a URI.

set-user!

instantiate

(Fn [(Ref URI a), (Maybe String)] ())

sets the user property of a URI in place.

str

defn

(Fn [(Ref URI a)] String) 

                        (str u)

prints the URL u as idempotently as possible, i.e. as the parsed string.

(def uri (URI.parse "http://admin:password@foo.com"))
(URI.str &uri) ; => "http://admin:password@foo.com"

unescape

defn

(Fn [(Ref String a)] String) 

                        (unescape s)

URI unescapes the string s, decoding each percent-triplet into a single byte and copying all other bytes through verbatim. UTF-8 characters that were percent-encoded byte by byte are therefore reassembled correctly.

update-fragment

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the fragment property of a URI using a function f.

update-host

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the host property of a URI using a function f.

update-opaque

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the opaque property of a URI using a function f.

update-password

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the password property of a URI using a function f.

update-path

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the path property of a URI using a function f.

update-port

instantiate

(Fn [URI, (Ref (Fn [(Maybe Int)] (Maybe Int) a) b)] URI)

updates the port property of a URI using a function f.

update-query

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the query property of a URI using a function f.

update-scheme

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the scheme property of a URI using a function f.

update-user

instantiate

(Fn [URI, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] URI)

updates the user property of a URI using a function f.

user

instantiate

(Fn [(Ref URI a)] (Ref (Maybe String) a))

gets the user property of a URI.

userinfo

defn

(Fn [(Ref URI a)] String) 

                        (userinfo u)

returns the user-information component for a URI u, which contains the provided username and password.

(def uri (URI.parse "http://admin:password@foo.com"))
(URI.userinfo &uri) ; => admin:password

zero

defn

(Fn [] URI) 

                        (zero)

Creates an empty URI, i.e. one that has only Nothing in it.