%html([wapp-debug-env])} } wapp-start $argv Many Wapp applications contain an /env page for debugging and trouble-shooting purpose. Examples: * https://sqlite.org/src/ext/checklist/top/env * https://sqlite.org/search?env=1 3.1 Binary Resources -------------------- Here is another variation on the same "hello, world" program that adds an image to the main page: > package require wapp proc wapp-default {} { set B [wapp-param BASE_URL] wapp-trim {
Hello, World!
See the Wapp Environment
Broccoli: 
Wapp Environment
\n
%html([wapp-debug-env])
}
}
proc wapp-page-broccoli.gif {} {
wapp-mimetype image/gif
wapp-cache-control max-age=3600
wapp-unsafe [binary decode base64 {
R0lGODlhIAAgAPMAAAAAAAAiAAAzMwBEAABVAABmMwCZMzPMM2bMM5nMM5nMmZn/
mczMmcz/mQAAAAAAACH5BAEAAA4ALAAAAAAgACAAAAT+0MlJXbmF1M35VUcojNJI
dh5YKEbRmqthAABaFaFsKG4hxJhCzSbBxXSGgYD1wQw7mENLd1FOMa3nZhUauFoY
K/YioEEP4WB1pB4NtJMMgTCoe3NWg2lfh68SCSEHP2hkYD4yPgJ9FFwGUkiHij87
ZF5vjQmPO4kuOZCIPYsFmEUgkIlJOVcXAS8DSVoxB0xgA6hqAZaksiCpPThghwO6
i0kBvb9BU8KkASPHfrXAF4VqSgAGAbpwDgRSaqQXrLwDCF5CG9/hpJKkb17n6RwA
18To7whJX0k2NHYjtgXoAwCWPgMM+hEBIFDguDrjZCBIOICIg4J27Lg4aGCBPn0/
FS1itJdNX4OPChditGOmpIGTMkJavEjDzASXMFPO7IAT5M6FBvQtiPnTX9CjdYqi
cFlgoNKlLbbJfLqh5pAIADs=
}]
}
wapp-start $argv
This application is the same as the previous except that it adds the
"broccoli.gif" image on the main "Hello, World" page. The image file is
a separate resource, which is provided by the new "wapp-page-broccoli.gif"
proc. The image is a GIF which has been encoded using base64 so that
it can be put into an text TCL script. The "`[binary decode base64 ...]`"
command is used to convert the image back into binary before returning
it.
Other resources might be added using procs like "wapp-page-style.css"
or "wapp-page-script.js".
4.0 General Structure Of A Wapp Application
-------------------------------------------
Wapp applications all follow the same basic template:
>
package require wapp;
proc wapp-page-XXXXX {} {
# code to generate page XXXXX
}
proc wapp-page-YYYYY {} {
# code to generate page YYYYY
}
proc wapp-default {} {
# code to generate any page not otherwise
# covered by wapp-page-* procs
}
wapp-start $argv
The application script first loads the Wapp code itself using
the "package require" at the top. (Some applications may choose
to substitute "source wapp.tcl" to accomplish the same thing.)
Next the application defines various procs that will generate the
replies to HTTP requests. Different procs are invoked based on the
first element of the URI past the Wapp script name. Finally,
the "wapp-start" routine is called to start Wapp running. The
"wapp-start" routine never returns (or in the case of CGI, it only
returns after the HTTP request has been completely processed),
so it should be the very last command in the application script.
4.1 Wapp Applications As Model-View-Controller
----------------------------------------------
If you are accustomed to thinking of web applications using the
Model-View-Controller (MVC) design pattern, Wapp supports that
point of view. A basic template for an MVC Wapp application
is like this:
>
package require wapp;
# procs to implement the model go here
proc wapp-page-XXXXX {} {
# code to implement controller for XXXXX
# code to implement view for XXXXX
}
proc wapp-page-YYYYY {} {
# code to implement controller for YYYYY
# code to implement view for YYYYY
}
proc wapp-default {} {
# code to implement controller for all other pages
# code to implement view for all other pages
}
wapp-start $argv
The controller and view portions of each page need not be coded
together into the same proc. They can each be sub-procs that
are invoked from the main proc, if separating the functions make
code clearer.
So Wapp does support MVC, but without a lot of complex
machinery and syntax.
5.0 Parameters
--------------
The purpose of a Wapp invocation is to answer an HTTP request.
That HTTP request is described by various "parameters".
Each parameter has a key and a value.
The Wapp application retrieves the value for the parameter with
key _NAME_ using a call to `[wapp-param` _NAME_`]`.
If there is no parameter with the key _NAME_, then the wapp-param
function returns an empty string.
Or, if wapp-param is given a second argument, the value of the second
argument is returned if there exists no parameter with a key of _NAME_.
5.1 Parameter Types
-------------------
There are four source of parameter data:
1. **CGI Parameters**
Parameters with upper-case names contain information about the
HTTP request as it was received by the web server. Examples of
CGI parameters are CONTENT\_LENGTH which is the number of bytes
of content in the HTTP request, REMOTE\_ADDR which holds the IP
address from which the HTTP request originated, REQUEST\_URI which
is the path component of the URL that caused the HTTP request,
and many others. Many of the CGI Parameters have names that
are the same as the traditional environment variables used to
pass information into CGI programs - hence the name "CGI Parameters".
However, with Wapp these values are not necessarily environment
variables and they all exist regardless of whether the application
is run using CGI, via SCGI, or using the built-in web server.
2. **Cookies**
If the HTTP request contained cookies, Wapp automatically decodes
the cookies into new Wapp parameters.
Only cookies that have lower-case names are decoded. This
prevents a cookie name from colliding with a CGI parameter.
Cookies that have uppercase letters in their name are silently
ignored.
3. **Query Parameters**
Query parameters are the key/value arguments that follow the "?"
in the URL of the HTTP request. Wapp automatically decodes the
key/value pairs and makes a new Wapp parameter for each one.
Only query parameter that have lower-case names are decoded. This
prevents a query parameter from overriding or impersonating a
CGI parameter. Query parameter with upper-case letters in their
name are silently ignored. Furthermore, query parameters are only
decoded if the HTTP request uses the same origin as the application,
or if the "wapp-allow-xorigin-params" has been run to signal Wapp
that cross-origin query parameters are allowed.
4. **POST Parameters**
POST parameters are the application/x-www-form-urlencoded key/value
pairs in the content of a POST request that typically originate from
forms. POST parameters are treated exactly like query parameters in
that they are decoded to form new Wapp parameters as long as they
have all lower-case keys and as long as either the HTTP request comes
from the same origin or the "wapp-allow-xorigin-params" command has
been run.
All Wapp parameters are held in a single namespace. There is no way to
distinguish a cookie from a query parameter from a POST parameter. CGI
parameters can be distinguished from the others by having all upper-case
names.
5.2 Parameter Examples
----------------------
To better understand how parameters work in Wapp, run the
"`env.tcl`" sample application in the Wapp source tree
(https://wapp.tcl.tk/home/file/examples/env.tcl). Like this:
>
wapptclsh examples/env.tcl
The command above should cause a web page to pop up in your web browser.
That page will look something like this:
>**Wapp Environment**
>
BASE_URL = http://127.0.0.1:33999
DOCUMENT_ROOT = /home/drh/wapp/examples
HTTP_ACCEPT_ENCODING = {gzip, deflate}
HTTP_COOKIE = {env-cookie=simple}
HTTP_HOST = 127.0.0.1:33999
HTTP_USER_AGENT = {Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0}
PATH_HEAD = {}
PATH_INFO = {}
PATH_TAIL = {}
QUERY_STRING = {}
REMOTE_ADDR = 127.0.0.1
REMOTE_PORT = 53060
REQUEST_METHOD = GET
REQUEST_URI = /
SAME_ORIGIN = 0
SCRIPT_FILENAME = /home/drh/wapp/examples/env.tcl
SCRIPT_NAME = {}
SELF_URL = http://127.0.0.1:33999/
WAPP_MODE = local
env-cookie = simple
[pwd] = /home/drh/wapp
Try this. Then modify the URL by adding new path elements and query
parameters to see how this affects the Wapp parameters.
Notice in particular how query parameters are decoded and added to the
set of Wapp parameters.
5.3 Parameter Security
----------------------
Parameter values in the original HTTP request may be encoded in various
ways. Wapp decodes parameter values before returning them to the
application. Application developers never see the encoded values.
There is never an opportunity to miss a decoding step.
For security reasons, Query and POST parameters are only added to the
Wapp parameter set if the inbound request is from the "same origin" or
if the special "wapp-allow-xorigin-params" interface is called.
An inbound request is from the same origin if it is in response to
clicking on a hyperlink or form on a page that was generated by the
same website.
Manually typing in a URL does not constitute the "same origin". Hence,
in the "env.tcl" example above the "wapp-allow-xorigin-params" interface
is used so that you can manually extend the URL to add new query parameters.
If query parameters can have side effects, then you should omit the
wapp-allow-xorigin-params call. The wapp-allow-xorigin-params command
is safe for read-only web pages. Do not invoke wapp-allow-xorigin-params
on pages where the parameters can be used to change server state.
5.4 CGI Parameter Details
-------------------------
The CGI parameters in Wapp describe the HTTP request that is to be answered
and the execution environment.
These parameter look like CGI environment variables. To prevent environment
information from overlapping and overwriting query parameters, all the
environment information uses upper-case names and all query parameters
are required to be lower case. If an input URL contains an upper-case
query parameter (or POST parameter or cookie), that parameter is silently
omitted.
The following CGI parameters are available:
+ **CONTENT\_LENGTH**
The number of bytes of POST data.
This parameter is either omitted or has a value of "0"
for non-POST requests.
+ **CONTENT\_TYPE**
The mimetype of the POST data. Usually this is
application/x-www-form-urlencoded.
This parameter is omitted for non-POST requests.
+ **DOCUMENT\_ROOT**
For CGI or SCGI, this parameter is the name a directory on the server
that is the root of the static content tree. When running a Wapp script
using the built-in web server, this is the name of the directory that
contains the script.
+ **HTTP\_COOKIE**
The values of all cookies in the HTTP header.
This parameter is omitted if there are no cookies.
+ **HTTP\_HOST**
The hostname (or IP address) and port that the client used to create
the current HTTP request. This is the first part of the request URL,
right after the "http://" or "https://". The format for this value
is "HOST:PORT". Examples: "sqlite.org:80" or "127.0.0.1:32172".
Some servers omit the port number if it has a value of 80.
+ **HTTP\_USER\_AGENT**
The name of the web-browser or other client program that generated
the current HTTP request, as reported in the User-Agent header.
+ **HTTPS**
If the HTTP request arrived of SSL (via "https://"), then this variable
has the value "on". For an unencrypted request ("http://"), this
parameter is undefined.
+ **PATH\_INFO**
The part of the URL path that follows the SCRIPT\_NAME. For all modes
other than CGI, this is exactly the URL pathname, though with the
query parameters removed. PATH_INFO begins with a "/".
+ **REMOTE\_ADDR**
The IP address from which the HTTP request originated.
+ **REMOTE\_PORT**
The TCP port from which the HTTP request originated.
+ **REQUEST\_METHOD**
"GET" or "HEAD" or "POST"
+ **REQUEST\_URI**
The URL for the inbound request, without the initial "http://" or
"https://" and without the HTTP\_HOST. This variable is the same as
the concatenation of $SCRIPT\_NAME and $PATH\_INFO.
+ **SCRIPT\_FILENAME**
The full pathname on the server for the Wapp script. This parameter
is usually undefined for SCGI.
+ **SCRIPT\_NAME**
In CGI mode, this is the name of the CGI script in the URL. In other
words, this is the initial part of the URL path that identifies the
CGI script. When using the built-in webserver, the value of this
parameter is an empty string. For SCGI, this parameter is normally
undefined.
All of the above are standard CGI environment values.
The following are supplemental environment parameters are added by Wapp:
+ **BASE\_URL**
The text of the request URL through the SCRIPT\_NAME. This value can
be prepended to hyperlinks to ensure that the correct page is reached by
those hyperlinks.
+ **CONTENT**
The raw POST data text.
+ **PATH\_HEAD**
The first element in the PATH\_INFO. The value of PATH\_HEAD is used to
select one of the "wapp-page-XXXXX" commands to run in order to generate
the output web page.
+ **PATH\_TAIL**
All of PATH\_INFO that follows PATH\_HEAD.
+ **SAME\_ORIGIN**
This value is either "1" or "0" depending on whether the current HTTP
request is a follow-on to another request from this same website or not.
Query parameters and POST parameters are usually only decoded and added
to Wapp's parameter list if SAME\_ORIGIN is 1. If a webpage implemented
by Wapp needs access to query parameters for a cross-origin request, then
it should invoke the "wapp-allow-xorigin-params" interface to explicitly
signal that cross-origin parameters are safe for that page.
+ **SELF\_URL**
The URL for the current page, stripped of query parameter. This is
useful for filling in the action= attribute of forms.
+ **SERVER\_ADDR**
In SCGI mode only, this variable is the address of the webserver from which
the SCGI request originates.
+ **WAPP\_MODE**
This parameter has a value of "cgi", "local", "scgi", or "server" depending
on how Wapp was launched.
### 5.4.1 URL Parsing Example
For the input URL "http://example.com/cgi-bin/script/method/extra/path?q1=5"
and for a CGI script named "script" in the /cgi-bin/ directory,
the following CGI environment values are generated:
+ **HTTP\_HOST** → "example.com:80"
+ **SCRIPT\_NAME** → "/cgi-bin/script"
+ **PATH\_INFO** → "/method/extra/path"
+ **REQUEST\_URI** → "/cgi-bin/script/method/extra/path"
+ **QUERY\_STRING** → "q1=5"
+ **BASE\_URL** → "http://example.com/cgi-bin/script"
+ **SELF\_URL** → "http://example.com/cgi-bin/script/method"
+ **PATH\_HEAD** → "method"
+ **PATH\_TAIL** → "extra/path"
The first five elements of the example above, HTTP\_HOST through
QUERY\_STRING, are standard CGI. The final four elements are Wapp
extensions. The following is the same information show in a diagram:
>
http://example.com/cgi-bin/script/method/extra/path?q1=5
\_________/\_____________/\________________/ \__/
| | | |
HTTP_HOST SCRIPT_NAME PATH_INFO `-- QUERY_STRING
>
http://example.com/cgi-bin/script/method/extra/path?q1=5
\_________/\_______________________________/ \__/
| | |
HTTP_HOST REQUEST_URI `-- QUERY_STRING
>
http://example.com/cgi-bin/script/method/extra/path?q1=5
\_______________________________/ \____/ \________/
| | |
BASE_URL PATH_HEAD PATH_TAIL
>
http://example.com/cgi-bin/script/method/extra/path?q1=5
\______________________________________/ \________/
| |
SELF_URL PATH_TAIL
### 5.4.2 Undefined Parameters When Using SCGI on Nginx
Some of the CGI parameters are undefined by default when using SCGI mode
with Nginx. If these CGI parameters are needed by the application, then
values must be assigned in the Nginx configuration file. For example:
>
location /scgi/ {
include scgi_params;
scgi_pass localhost:9000;
scgi_param SCRIPT_NAME "/scgi";
scgi_param SCRIPT_FILENAME "/home/www/scgi/script1.tcl";
}
6.0 Wapp Commands
-----------------
Wapp is really just a collection of TCL procs. All procs are in a single file
named "wapp.tcl".
The procs that form the public interface for Wapp begin with "wapp-". The
implementation uses various private procedures that have names beginning
with "wappInt-". Applications should use the public interface only.
The most important Wapp interfaces are:
+ **wapp-start**
+ **wapp-subst** and **wapp-trim**
+ **wapp-param**
Understand the four interfaces above, and you will have a good understanding
of Wapp. The other interfaces are merely details.
The following is a complete list of the public interface procs in Wapp:
+ **wapp-start** _ARGLIST_
Start up the application. _ARGLIST_ is typically the value of $::argv,
though it might be some subset of $::argv if the containing application
has already processed some command-line parameters for itself. By default,
this proc never returns, and so it should be very last command in the
application script. To embed Wapp in a larger application, include
the -nowait option in _ARGLIST_ and this proc will return immediately
after setting up all necessary file events.
+ **wapp-subst** _TEXT_
This command appends text to the end of reply to an HTTP request.
The _TEXT_ argument should be enclosed in {...} to prevent
accidental substitutions.
The "wapp-subst" command itself will do all necessary backslash
substitutions. Command and variable substitutions occur
within "%html(...)", "%url(...)", "%qp(...)", "%string(...)", and
"%unsafe(...)". The substitutions are escaped (except in the case of
"%unsafe(...)") so that the result is safe for inclusion within the
body of an HTML document, a URL, a query parameter, or a javascript or
JSON string literal, respectively.
> > Caution #1: When using Tcl 8.6 or
earlier, command substitution, but not variable substitution, occurs
outside of the quoted regions. This problem is fixed using the new
"-command" option to the regsub command in Tcl 8.7. Nevertheless,
it is suggested that you avoid using the "\[" character outside of
the %-quotes. Use "\[" instead.
> > Caution #2: The %html() and similar %-substitutions are parsed
using a regexp, which means that they cannot do matching parentheses.
The %-substitution is terminated by the first close parenthesis, not the
first matching close-parenthesis.
+ **wapp-trim** _TEXT_
Just like wapp-subst, this routine appends _TEXT_ to the web page
under construction. The same substitution functions
are supported. The difference is that this routine also removes
surplus whitespace from the left margin, so that if the _TEXT_
argument is indented in the source script, it will appear at the
left margin in the generated output.
+ **wapp-param** _NAME_ _DEFAULT_
Return the value of the Wapp parameter _NAME_,
or return _DEFAULT_ if there is no such query parameter or environment
variable. If _DEFAULT_ is omitted, then it is an empty string.
+ **wapp-set-param** _NAME_ _VALUE_
Change the value of parameter _NAME_ to _VALUE_. If _NAME_ does not
currently exist, it is created.
+ **wapp-param-exists** _NAME_
Return true if and only if a parameter called _NAME_ exists for the
current request.
+ **wapp-param-list** _NAME_
Return a TCL list containing the names of all parameters for the current
request. Note that there are several parameters that Wapp uses
internally. Those internal-use parameters all have names that begin
with ".".
+ **wapp-allow-xorigin-params**
Query parameters and POST parameters are usually only parsed and added
to the set of parameters available to "wapp-param" for same-origin
requests. This restriction helps prevent cross-site request forgery
(CSRF) attacks. Query-only web pages for which it is safe to accept
cross-site query parameters can invoke this routine to cause query
parameters to be decoded.
+ **wapp-mimetype** _MIMETYPE_
Set the MIME-type for the generated web page. The default is "text/html".
+ **wapp-reply-code** _CODE_
Set the reply-code for the HTTP request. The default is "200 Ok".
+ **wapp-redirect** _TARGET-URL_
Cause an HTTP redirect to _TARGET-URL_.
+ **wapp-reset**
Reset the web page under construction back to an empty string.
+ **wapp-set-cookie** _NAME_ _VALUE_
Cause the cookie _NAME_ to be set to _VALUE_.
+ **wapp-clear-cookie** _NAME_
Erase the cookie _NAME_.
+ **wapp-safety-check**
Examine all TCL procedures in the application and return a text string
containing warnings about unsafe usage of Wapp commands. This command
is run automatically if the "wapp-start" command is invoked with a --lint
option.
+ **wapp-cache-control** _CONTROL_
The _CONTROL_ argument should be one of "no-cache", "max-age=N", or
"private,max-age=N", where N is an integer number of seconds.
+ **wapp-content-security-policy** _POLICY_
Set the Content Security Policy (hereafter "CSP") to _POLICY_. The
default CSP is _default\_src 'self'_, which is very restrictive. The
default CSP disallows (a) loading any resources from other origins,
(b) the use of eval(), and (c) in-line javascript or CSS of any kind.
Set _POLICY_ to "off" to completely disable the CSP mechanism. Or
specify some other policy suitable for the needs of the application.
The following allows inline images using
<img src='data:...'> and inline "style='...'" attributes,
but restricts all other attack vectors and thus seems to be a good
choice for many applications:
wapp-content-security-policy {
default-src 'self' data:;
style-src 'self' 'unsafe-inline';
}
+ **wapp-debug-env**
This routine returns text that describes all of the Wapp parameters.
Use it to get a parameter dump for troubleshooting purposes.
+ **wapp** _TEXT_
Add _TEXT_ to the web page output currently under construction. _TEXT_
must not contain any TCL variable or command substitutions. This command
is rarely used.
+ **wapp-unsafe** _TEXT_
Add _TEXT_ to the web page under construction even though _TEXT_ does
contain TCL variable and command substitutions. The application developer
must ensure that the variable and command substitutions does not allow
XSS attacks. Avoid using this command. The use of "wapp-subst" is
preferred in most situations.
7.0 Security
------------
Wapp strives for security by default. Applications can disable security
features on an as-needed basis, but the default setting for security
features is always "on".
Security features in Wapp include:
1. The default Content Security Policy \[2\] or "CSP"
for all Wapp applications is _default-src 'self'_. In that mode,
resources must all be loaded from the same origin, the use of
eval() and similar commands in javascript is prohibited, and
no in-line javascript or CSS is allowed. These limitations help
keep applications safe from Cross-site Scripting or XSS attacks \[3\],
attacks, even in the face of application coding errors. If these
restrictions are too severe for an application, the CSP can be
relaxed or disabled using the
"`wapp-content-security-policy`" command.
2. Access to GET query parameters and POST parameters is prohibited
unless the origin of the request is the application itself, as
determined by the Referrer field in the HTTP header. This feature
helps to prevent
[Cross-site Request Forgery]\[4\]
attacks. The
"`wapp-allow-xorigin-params`" command
can be used to disable this protection on a case-by-case basis.
3. Cookies, query parameters, and POST parameters are automatically
decoded before they reach application code. There is no risk
that the application program will forget a decoding step or
accidently miscode a decoding operation.
4. Cookies, query parameters, and POST parameters are silently discarded
unless their names begin with a lower-case letter and contain only
alphanumerics, underscores, and minus-signs. Hence, there is no risk
that unusual parameter names can cause quoting problems or other
vulnerabilities.
5. Reply text generated using the "`wapp-subst`" and "`wapp-trim`" commands
automatically escapes generated text so that it is safe for inclusion
within HTML, within a javascript or JSON string literal, as a URL,
or as the value of a query parameter. As long as the application
programmer is careful to always use "wapp-subst" and/or "wapp-trim"
to generate replies, there is little risk of injection attacks.
6. If the application is launched on a command-line with the --lint
option, then instead of running the application, Wapp scans the
application code looking for constructs that are unsafe. Unsafe
constructs include things such as using
"\wapp-subst\" with an argument
that is not contained within {...}.
7. The new (non-standard) SAME\_ORIGIN variable is provided. This variable
has a value of "1" or "0" depending on whether or not the current HTTP
request comes from the same origin. Applications can use this information
to enhance their own security precautions by refusing to provide sensitive
information or perform sensitive actions if SAME\_ORIGIN is not "1".
8. The --scgi mode only accepts SCGI requests from localhost. This prevents
an attacker from sending an SCGI request directly to the script and bypassing
the webserver in the event that the site firewall is misconfigured or omitted.
9. Though cookies, query parameters and POST parameters are accessed using
the same mechanism as CGI variables, the CGI variable names use a disjoint
namespace. (CGI variables are all upper-case and all others are lower-case.)
Hence, it is not possible for a remote attacher to create a fake CGI variable
or override the value of a CGI variable.
Part of what makes Wapp easy to use is that it helps free application
developers from the worry of accidently introducing security vulnerabilities
via programming errors. Of course, no framework is fool-proof. Developers
still must be aware of security. Wapp does not prevent every error, but
it does help make writing a secure application easier and less stressful.
8.0 End Notes
-------------
1. https://sqlite.org/
2. https://en.wikipedia.org/wiki/Content\_Security\_Policy
3. https://en.wikipedia.org/wiki/Cross-site\_scripting
4. https://en.wikipedia.org/wiki/Cross-site\_request\_forgery