YAWS_API(5) User API YAWS_API(5)

yaws_api - api available to yaws web server programmers

yaws_api:Function(...)

This is the api available to yaws web server programmers. The Erlang module yaws_api contains a wide variety of functions that can be used inside yaws pages.

Each chunk of yaws code is executed while the yaws page is being delivered from the server. We give a very simple example here to show the basic idea. Imagine the following HTML code:

<html>
<body>
<h1> Header 1</h1>
<erl>
out(Arg) ->
    {html, "<p> Insert this text into the document"}.
</erl>
</body>
</html>

The out(Arg) function is supplied one argument, an #arg{} structure. We have the following relevant record definitions:

-record(arg, {
          clisock,        % the socket leading to the peer client
          client_ip_port, % {ClientIp, ClientPort} tuple
          headers,        % headers
          req,            % request (possibly rewritten)
          orig_req,       % original request
          clidata,        % The client data (as a binary in POST requests)
          server_path,    % The normalized server path
                          % (pre-querystring part of URI)
          querydata,      % For URIs of the form ...?querydata
                          %  equiv of cgi QUERY_STRING
          appmoddata,     % (deprecated - use pathinfo instead) the remainder
                          % of the path leading up to the query
          docroot,        % Physical base location of data for this request
          docroot_mount,  % virtual directory e.g /myapp/ that the docroot
                          %  refers to.
          fullpath,       % full deep path to yaws file
          cont,           % Continuation for chunked multipart uploads
          state,          % State for use by users of the out/1 callback
          pid,            % pid of the yaws worker process
          opaque,         % useful to pass static data
          appmod_prepath, % (deprecated - use prepath instead) path in front
                          %  of: <appmod><appmoddata>
          prepath,        % Path prior to 'dynamic' segment of URI.
                          %  ie http://some.host/<prepath>/<script-point>/d/e
                          % where <script-point> is an appmod mount point,
                          % or .yaws,.php,.cgi,.fcgi etc script file.
          pathinfo,       % Set to '/d/e' when calling c.yaws for the request
                          % http://some.host/a/b/c.yaws/d/e
                          %  equiv of cgi PATH_INFO
          appmod_name     % name of the appmod handling a request,
                          % or undefined if not applicable
         }).

The headers argument is also a record: 

-record(headers, {
          connection,
          accept,
          host,
          if_modified_since,
          if_match,
          if_none_match,
          if_range,
          if_unmodified_since,
          range,
          referer,
          user_agent,
          accept_ranges,
          cookie = [],
          keep_alive,
          location,
          content_length,
          content_type,
          content_encoding,
          authorization,
          transfer_encoding,
          x_forwarded_for,
          other = []   % misc other headers
         }).

The out/1 function can use the Arg to generate any content it likes. We have the following functions to aid that generation.

Server side include. Just include the files as is in the document. The files will not be parsed and searched for <erl> tags.

Server side include of pre-indented code. The data in Files will be included but contained in a <pre> tag. The data will be htmlized.

Include htmlized content from String.

The equivalent of io_lib:format/2. This function is automatically -included in all erlang code which is a part of a yaws page.

Htmlize an IO list object.

Sets a cookie to the browser. Options are: 
{expires, UtcTime}  - Cookie expiration time, where UtcTime is
                      a tuple returned by calendar:universal_time/0.
{max_age, Age}      - Defines the lifetime of the cookie, in seconds,
                      where age is an integer >= 0.
{path, Path}        - Path is a string that specifies the subset of URLs to
                      which this cookie applies.
{domain, Domain}    - Domain is a string that specifies the domain for which
                      the cookie is valid.
{same_site, Policy} - Policy is one of the atoms lax, none or strict.
{comment, Comment}  - Comment is a string that doccuments the server's
                      intended use of the cookie.
secure              - Directs the user agent to use only secure means to
                      contact the origin server whenever it sends back this
                      cookie.
http_only           - Restricts cookie access from other non-HTTP APIs.

Sets a cookie to the browser. This function is deprecated by set_cookie/3.

This function can be used to search for a cookie that was previously set by setcookie/2-6. For example if we set a cookie as yaws_api:setcookie("sid",SomeRandomSid), then on subsequent requests from the browser we can call: find_cookie("sid",(Arg#arg.headers)#headers.cookie)

The function returns [] if no cookie was found, otherwise the actual cookie is returned as a string.

This function parses the value of a Set-Cookie header, following the RFC6265. Because old RFCs (2109 and 2965) are still used, it is backward compatible. So this function returns a #setcookie{} record when only one cookie is found. If multiple cookies are set in a single Set-Cookie header, it returns a list of #setcookie{} records. If no cookie was found or if an error occurred, it returns [].

#setcookie{} record is defined in yaws_api.hrl: 

-record(setcookie, {key,
                    value,
                    quoted = false,
                    domain,
                    max_age,
                    expires,
                    path,
                    secure = false,
                    http_only = false,
                    extensions = []}).

This function parses the value of Cookie header, following the RFC6265. It returns a list of #cookie{} records. If no cookie was found or if an error occurred, it returns [].

#cookie{} record is defined in yaws_api.hrl: 

-record(cookie, {key,
                 value,
                 quoted = false}).

Build a cookie string from a #setcookie{} record like returned by parse_set_cookie/1.

Build a cookie string from a #cookie{} record (or a list or records) like returned by parse_cookie/1.

This function generates a redirect to the browser. It will clear any previously set headers. So to generate a redirect and set a cookie, we need to set the cookie after the redirect as in: 
out(Arg) ->
  ... do some stuff
  Ret = [{redirect, "http://www.somewhere.com"},
          setcookie("sid", Random)
        ].

If we want to issue a redirect to ourselves, this function is useful. It returns a record #redir_self{} defined in yaws_api.hrl. The record contains fields to construct a URL to ourselves. 
-record(redir_self, {
          host,        % string() - our own host
          scheme,      % http | https
          scheme_str,  % "https://"  | "http://"
          port,        % integer()  - our own port
          port_str     % "" | ":<int>" - the optional port part
                       %                 to append to the url
         }).

This function is convenient when getting \r\n terminated lines from a stream of data. It returns:

{line, Line, Tail} or {lastline, Line, Tail}

The function handles multilines as defined in e.g. SMTP or HTTP

Returns the MIME type as defined by the extension of FileName. Scope can have following values:

global - returns the result obtained from the global context.
#sconf{} | {ServerName, Port} - returns the result obtained from the virtual server's context. If no MIME type is found in this scope, it falls back on the global one.

Tries to determine the right Scope before calling mime_type/2.

When a yaws function needs to deliver chunks of data which it gets from a process. The other process can call this function to deliver these chunks. It requires the out/1 function to return the value {streamcontent, MimeType, FirstChunk} to work. YawsPid is the process identifier of the yaws process delivering the original .yaws file. That is self() in the yaws code. The Pid must typically be passed (somehow) to the producer of the stream.

A synchronous version of the above function. This synchronous version must always be used when the producer of the stream is faster than the consumer. This is usually the case since the client is the WWW browser.

When the process discussed above is done delivering data, it must call this function to let the yaws content delivering process finish up the HTTP transaction.

Yaws allows application processes to deliver data directly to the client. The application tells yaws about such a process by returning {streamcontent_from_pid, MimeType, Pid} from its out/1 function. In this case, Pid uses the stream_process_deliver/2 function to deliver data to the client. The application gets Socket from Arg#arg.clisock, and IoList is the data to be sent to the client.

Same as above but delivers IoList using HTTP chunked transfer format. IoList must have a size greater than zero. The application process delivering the data will have had to have make sure that the HTTP headers of the response indicate chunked transfer mode, either by ensuring no Content-Length header is set or by specifically setting the Transfer-Encoding header to chunked.

If the application process delivering data to the client uses chunked transfer mode, it must call this to deliver the final chunk of the transfer. This tells yaws to create a special final chunk in the format required by the HTTP specification (RFC 2616). IoList may be empty, but if its size is greater than zero, that data will be sent as a separate chunk before the final chunk.

Application processes delivering data directly to clients must call this function to inform yaws that they've finished using Socket. The YawsPid argument will have been passed to the process earlier when yaws sent it a message telling it to proceed with data delivery. Yaws expects Socket to be open.

Same as the previous function but the application calls this if it closes the client socket as part of its data delivery process. This allows yaws to continue without assuming the socket is still open and encountering errors due to that assumption. The YawsPid argument will have been passed to the application process earlier when yaws sent it a message telling it to proceed with data delivery.

This function will parse the query part of the URL. It will return a {Key, Value} list.

This function is automatically included from yaws_api in all .yaws pages. It is used to search for a variable in the querypart of the url. Returns {ok, Val} or undefined. If a variable is defined multiple times, the function may also return {Val1, Val2...}.

If the browser has set the Content-Type header to the value "application/x-www-form-urlencoded", this function will parse the request's body. It will return a {Key, Value} list.

This function is automatically included from yaws_api in all .yaws pages. It is used to search for a variable in the request's body sent by the client. Returns {ok, Val} or undefined. If a variable is defined multiple times, the function may also return {Val1, Val2...}.

This function is used to search a variable in the query part of the URL and in the request's body. it invokes queryvar/2 and postvar/2 and merges the results.

If the browser has set the Content-Type header to the value "multipart/form-data", which is the case when the browser wants to upload a file to the server the following happens:

If the function returns {result, Res} no more data will come from the browser.

If the function returns {cont, Cont, Res} the browser will supply more data. (The file was too big to come in one read)

This indicates that there is more data to come and the out/1 function should return {get_more, Cont, User_state} where User_state might usefully be a File Descriptor. The Res value is a list of either: {head, {Name, Headers}} | {part_body, Binary} | {body, Binary}

The function returns {error, Reason} when an error occurred during the parsing.

Example usage could be: 

 <erl>
 out(A) ->
        case yaws_api:parse_multipart_post(A) of
             {cont, Cont, Res} ->
                    St = handle_res(A, Res),
                    {get_more, Cont, St};
             {result, Res} ->
                    handle_res(A, Res),
                    {html, f("<pre>Done </pre>",[])};
             {error, Reason} ->
                    {html, f("An error occured: ~p", [Reason])}
        end.
 handle_res(A, [{head, {Name, _Hdrs}}|T]) ->
      io:format("head:~p~n",[Name]),
      handle_res(A, T);
 handle_res(A, [{part_body, Data}|T]) ->
      io:format("part_body:~p~n",[Data]),
      handle_res(A, T);
 handle_res(A, [{body, Data}|T]) ->
      io:format("body:~p~n",[Data]),
      handle_res(A, T);
 handle_res(A, []) ->
      io:format("End_res~n").
 </erl>

Create a new cookie-based session. Yaws will either generate the cookie itself or, if a ysession_cookiegen module is configured, call new_cookie() on that module to get a new cookie. The new cookie is returned from this function. The Opaque argument will typically contain user data such as user name and password

As above, but allows to set a session specific time-out value, overriding the system specified time-out value.

As above, but also sends a message {yaws_session_end, Reason, Cookie, Opaque} to the provided CleanupPid where Reason can be either of timeout or normal. The Cookie is the HTTP cookie as returned by new_session() and Opaque is the user-provided Opaque parameter to new_session(). The purpose of the feature is to cleanup resources assigned to the session.

This function is intended for embedded mode in yaws. It makes it possible to load a yaws configuration from another data source than /etc/yaws.conf, such as a database. If yaws is started with the environment {embedded, true}, yaws will start with an empty default configuration, and wait for some other program to execute a setconf/2 The Gconf is a #gconf{} record and the Group variable is a list of lists of #sconf{} records. Each sublist must contain #sconf{} records with the same IP/Port listen address. To create a suitable initial #gconf{} record see the code in yaws_config:make_default_gconf/2. Especially the yaws_dir parameter is important to get right.

Decode url-encoded string. A URL encoded string is a string where all alfa numeric characters and the the character _ are preserved and all other characters are encode as "%XY" where X and Y are the hex values of the least respective most significant 4 bits in the 8 bit character.

URL-encodes a string or binary, and returns a string. All URLs in HTML documents must be URL encoded.

Returns a socket for SSL sockets or the atom undefined for non-SSL sockets. Useful for applications that have to deal with both SSL and non-SSL sockets.

Return the actual port number used by the listen socket of the virtual server indicated by the function argument, an #sconf{} record instance. If successful, returns the requested port number, or returns {error, not_found} if the function argument does not match any known virtual server. This function is useful for retrieving the actual port number when, e.g. for testing purposes, a virtual server is configured to use port 0, which will cause it to have an ephemeral port assigned by the operating system.

Returns a list of reformatted header values from a #headers{} record. The return list is suitable for retransmit.

Returns a list of reformatted header values from a #headers{} record, with each element of the list formatted via a call to FormatFun. This enables converting #headers{} records into various lists of headers and their values. Note that sometimes the Set-Cookie header or other headers will contain a tuple value of the form {multi, ValueList}. (The {multi, ValueList} construct typically results from calls to merge_header/2 or merge_header/3, where multiple values are set in separate calls for the same header; see merge_header/2 below for details.) Formatting functions should therefore be capable of handling a {multi, ValueList} tuple. They should handle it by formatting each member of ValueList as a separate header string, storing all such header strings in a list, and returning that list in a {multi, HdrList} tuple. Note that in versions of Yaws 2.0.6 and older, formatting functions returned such header lists directly, which implies that sometimes the return values of reformat_header/1 and reformat_header/2 can be a multi-level list if constructed by one of these older formatting functions.

Same as reformat_header/2 except that header and value data passed to FormatFun are first converted to the data format specified in Options. Options is expected to be either an atom or a list of atoms, either string or binary. If the list contains multiple items, options earlier in the list override those later in the list, so for example [string, binary] is the same as [string]. If the first item in Options specifies anything other than string or binary, data are passed to FormatFun without conversion.

Sets header Header with value Value in the #headers{} record Headers, and returns a new #headers{} record. Using the atom undefined for Value effectively deletes the header, same as delete_header/2.

Same as set_header/2 above, except Header and Value are not passed in a tuple.

Merges value Value for header Header with any existing value for that header in the #headers{} record Headers, and returns a new #headers{} record. Using the atom undefined for Value simply returns Headers. Otherwise, Value is merged with any existing value already present in the Headers record for header Header, comma-separated from that existing value. If no such value exists in the Headers record, the effect is the same as set_header/2. Note that for the Set-Cookie header, values are not comma-separated but are instead collected into a tuple {multi, ValueList} where ValueList is the collection of Set-Cookie values. This implies that any formatting fun passed to reformat_header/2 must be prepared to handle such tuples.

Same as merge_header/2 above, except Header and Value are not passed in a tuple.

Gets the value of header Header from the #headers{} record Headers and returns it. If the header isn't set, the atom undefined is returned.

Deletes any value set for header Header in the #headers{} record Headers, and returns a new #headers{} record.

Return the url as requested by the client. Return value is a #url{} record as defined in yaws_api.hrl

Parse URL in a string, returns a #url record

Takes a #url record a formats the Url as a string

Calls an executable CGI script, given by its full path. Used to make `.yaws' wrappers for CGI programs. This function usually returns streamcontent.

Like before, but calls Exefilename to handle the script. The file name of the script is handed to the executable via a CGI meta variable.

Calls a FastCGI responder. The address and port of the FastCGI application server are taken from the server configuration (see yaws.conf). Used to make `.yaws' wrappers for FastCGI responders. Returns the same return values as out/1 (see below).

Same as above, but Options overrides the defaults from the server configuration: 

Options = [Option]
Option -- one of the following:

{app_server_host, string() | ip_address()} The hostname or the IP address of the FastCGI application server.

{app_server_port, 0..65535} The TCP port number of the FastCGI application server.

{path_info, string()} Override default pathinfo in Arg#arg.pathinfo.

{extra_env, ExtraEnv} Extra environment variables to be passed to the FastCGI application server, as a list of name-value pairs.

ExtraEnv = [Var]
Var = {Name, Value}
Name = string() | binary()
Value = string() | binary()

{trace_protocol, boolean()} Enable or disable tracing of FastCGI protocol messages as info log messages.

{log_app_error, boolean()} Enable or disable logging of application error messages: output to stderr and non-zero exit value.

Calls a FastCGI authorizer. The address and port of the FastCGI application server are taken from the server configuration (see yaws.conf). Used to make `.yaws' wrappers for FastCGI authorizers. Variables contains the values of the variables returned by the FastCGI application server in the "Variable-XXX: YYY" headers.

If access is denied, Out contains the complete response returned by the FastCGI application server. This response is typically returned as-is to the HTTP client.

If access is allowed, Out contains the response returned by the FastCGI application server minus the body (i.e. minus the content) which should be ignored per the FastCGI specification. This response is typically not returned to the HTTP client. The calling application module may wish to inspect the response, for example by extracting variables (see fcgi_extract_variables below) or by inspecting the headers returned by the FastCGI application server.

Out -- See return values for out/1 below

Same as above, but Options overrides the defaults from the server configuration. See call_fcgi_responder/2 above for a description of Options.

Extracts the environment variables from a FastCGI authorizer response by looking for headers of the form "Variable-Name: Value". 

Name = string() -- The name of the variable (the "Variable-" prefix
has already been removed).
Value = string() -- The value of the variable.

Perform a directory listing. Can be used in special directories when we don't want to turn on dir listings for the entire server. Always returns ok.

The out/1 function can return different values to control the behavior of the server.

{html, DeepList}
This assumes that DeepList is formatted HTML code. The code will be inserted in the page.

{ehtml|exhtml, Term}
This will transform the erlang term Term into a stream of HTML content. The exhtml variant transforms into strict XHTML code. The basic syntax of Term is 

EHTML = [EHTML] | {Tag, Attrs, Body} | {Tag, Attrs} | {Tag} |
        {Module, Fun, [Args]} | fun/0 |
        binary() | character()
Tag   = atom()
Attrs = [{Key, Value}]
Key   = atom()
Value = string() | binary() | atom() | integer() | float() |
        {Module, Fun, [Args]} | fun/0
Body  = EHTML

For example, {p, [], "Howdy"} expands into "<p>Howdy</p>" and

{form, [{action, "a.yaws"}],
   {input, [{type,text}]}}

expands into

<form action="a.yaws"
  <input type="text">
</form>

It may be more convenient to generate erlang tuples than plain html code.

{content, MimeType, Content}
This function will make the web server generate different content than HTML. This return value is only allowed in a yaws file which has only one <erl> </erl> part and no html parts at all.

{streamcontent, MimeType, FirstChunk}
This return value plays the same role as the content return value above.

However it makes it possible to stream data to the client if the yaws code doesn't have access to all the data in one go. (Typically if a file is very large or if data arrives from back end servers on the network.

{streamcontent_with_timeout, MimeType, FirstChunk, Timeout}
Similar to above, but with an explicit timeout. The default timeout is 30 secs. I.e if the application fails to deliver data to the Yaws process, the streaming will stop. This is often not the desired behaviour in Comet/Ajax applications. It's possible to provide 'infinity' as timeout.

{streamcontent_from_pid, MimeType, Pid}
This return value is similar to the streamcontent return value above.

However it makes it possible to stream data to the client directly from an application process to the socket. This approach can be useful for applications that employ long-polling (Comet) techniques, for example, and for applications wanting to avoid buffering data or avoid HTTP chunked mode transfer for streamed data.

{streamcontent_with_size, Sz, MimeType, FirstChunk}
This return value is similar to the streamcontent return value above.

However it makes it possible to stream data to the client by setting the content length of the response. As the opposite of other ways to stream data, in this case, the response is not chunked encoded.

{header, H}
Accumulates a HTTP header. The trailing CRNL which is supposed to end all HTTP headers must NOT be added. It is added by the server. The following list of headers are given special treatment.

{connection, What}

This sets the Connection: header. If What is the special value "close", the connection will be closed once the yaws page is delivered to the client.

{server, What}

Sets the Server: header. By setting this header, the server's signature will be dynamically overloaded.

{location, Url}

Sets the Location: header. This header is typically combined with the {status, 302} return value.

{cache_control, What}

Sets the Cache-Control: header.

{expires, What}

Sets the Expires: header.

{date, What}

Sets the Date: header.

{allow, What}

Sets the Allow: header.

{last_modified, What}

Sets the Last-Modified: header.

{etag, What}

Sets the Etag: header.

{set_cookie, Cookie}

Prepends a Set-Cookie: header to the list of previously set Set-Cookie: headers.

{content_range, What}

Sets the Content-Range: header.

{content_type, MimeType}

Sets the Content-Type: header.

{content_encoding, What}

Sets the Content-Encoding: header. If this header is defined, no deflate is performed by Yaws, allowing you to compress data yourself if you wish to do so.

{content_length, Len}

Normally yaws will ship Yaws pages using Transfer-Encoding: chunked. This is because we generally can't know how long a yaws page will be. If we for some reason want to force a Content-Length: header (and we actually do know the length of the content, we can force Yaws to not ship the page chunked.

{transfer_encoding, What}

Sets the Transfer-Encoding: header.

{www_authenticate, What}

Sets the WWW-Authenticate: header.

{vary, What}

Sets the Vary: header.

{accept_ranges, What}

Sets the Accept-Ranges: header.

All other headers must be added using the normal HTTP syntax. Example:

{header, {"My-X-Header", "gadong"}} or {header, "My-X-Header: gadong"}

{header, {HeaderName, erase}}
Clears the header named HeaderName from the accumulated headers.

{allheaders, HeaderList}
Will clear all previously accumulated headers and replace them.

{status, Code}
Will set another HTTP status code than 200.

Will stop processing of any consecutive chunks of erl or html code in the yaws file.

Do nothing.

Flush remaining data sent by the client.

{redirect, Url}
Erase all previous headers and accumulate a single Location header. Set the status code.

{redirect_local, Path}
Does a redirect to the same Scheme://Host:Port/Path as we currently are executing in.

{get_more, Cont, State}
When we are receiving large POSTs we can return this value and be invoked again when more Data arrives.

{page, Page}

Make Yaws returns a different page than the one being requested. Page is a Request-URI, so it must be url-encoded and can contain a query-string.

{page, {Options, Page}}
Like the above, but supplying an additional deep list of options. Supported option types are:

{status, C} - Set the HTTP response status code C for page Page.

{header, H} - Accumulate the HTTP header H for page Page.

{disable_cache, Bool} - if set to true, disable the cache of Page for this call.

{websocket, CallbackModule, Options}
Tell Yaws to use CallbackModule as a WebSocket Protocol handler for traffic on the client socket. See the Yaws websocket documentation for more details.

{ssi, File, Delimiter, Bindings}
Server side include File and macro expansion in File. Each occurrence of a string, say "xyz", inside File that's within a Delimiter pair is replaced with the corresponding value in Bindings.

Example: Delimiter = %%

File contains the string .... %%xyz%% .....

Bindings contain the tuple {"xyz", "Dingbat"}

The occurrence of %%xyz%% in File will be replaced with "Dingbat" in the Server side included output.

The {ssi, File, Delimiter, Bindings} statement can also occur within a deep ehtml structure.

The special directive strip_undefined can be specified in the Bindings list, just as it can for the {bindings, ....} directive, but it's ignored because treating undefined variables as empty is the default for ssi bindings.

{bindings, [{Key1, Value2}, {Key2, Value2} .....]}
Establish variable bindings that can be used in the page.

All bindings can then be used in the rest of yaws code (in HTML source and within erl tags). In HTML source %%Key%% is expanded to Value and within erl tags yaws_api:binding(Key) (which calls error if no such binding exists) or yaws_api:binding_find(Key) (which returns undefined if no such binding exists) can be used to extract Value, and yaws_api:binding_exists(Key) can be used to check for the existence of a binding.

If a page happens to contains text that looks like a binding, e.g. %%SomeText%%, but no key SomeText is supplied, then by default the original text is left as is. If you prefer that anything parsed as a binding gets stripped out of a page whenever the bindings directive does not specify its key, include the special directive strip_undefined in the bindings list:

{bindings, [{Key1, Value1}, strip_undefined]}

{yssi, YawsFile}
Include a yaws file. Compile it and expand as if it had occured inline.

#arg{}
Return an instance of an #arg{} record. This can be useful when used as part of a [ListOfValues] return value, so that any subsequent elements in the return list that require an #arg{} get the returned instance rather than the original. For example, an out/1 function might set the state field of an #arg{}, then return both it and {yssi, YawsFile} in a list, in which case Yaws will pass the returned #arg{}, rather than the original instance, to the yaws file out/1 function.

[ListOfValues]
It is possible to return a deep list of the above defined return values. Any occurrence of streamcontent, streamcontent_with_timeout, streamcontent_with_size, streamcontent_from_pid, get_more, page or break in this list is legal only if it is the last position of the list. If not, remaining values in the list are ignored.

Written by Claes Wikstrom

yaws.conf(5) erl(1)