<section id="lazybridge">
    <title>Example: the <quote>Lazy</quote> bridge</title>
	<section>
	<title>The rationale of threaded bridges</title>
    <para>    	
        The 'bridge' concept was introduced to cope with the ability of 
        the Apache HTTP web server to adopt different multiprocessing 
        models by loading one of the available MPMs (Multi Processing Modules). 
        A bridge's task is to let mod_rivet fit the selected multiprocessing
        model in the first place. Still separating mod_rivet core
        functions from the MPM machinery provided also a solution for
        implementing a flexible and extensible design that enables 
        a programmer to develop alternative approaches to workload and 
        resource management. 
   	</para>
   	<para>
   		The Apache HTTP web server demands its modules to
   		run with any MPM irrespective of its internal architecture and its
   		a general design constrain to make no assumptions about the MPM. 
   		This clashes with some requirements of threaded builds of Tcl. 
   		First of all Tcl is itself threaded (unless threads are disabled 
   		at compile time) and many of the basic Tcl data structures (namely Tcl_Obj) 
   		cannot be safely shared among threads. 
   		This demands a Tcl interpreters be run 
   		on separated threads communicating with the HTTP web server 
   		through suitable methods.
   	</para>
	</section>
	<section>
	<title>Lazy bridge data structures</title>
    <para>
       The lazy bridge was initially developed to outline the basic tasks carried
        out by each function making a rivet MPM bridge. The lazy bridge attempts
        to be minimalist but it's nearly fully functional, only a few configuration
        directives (SeparateVirtualInterps and SeparateChannel) are ignored because
        fundamentally incompatible. The bridge is experimental but perfectly fit
        for many applications, for example it's good on development machines where
        server restarts are frequent. 
    </para>

    <para>
    	This is the lazy bridge jump table, as such it defines the functions
    	implemented by the bridge.
    </para>
    
    <programlisting>RIVET_MPM_BRIDGE {
    LazyBridge_ServerInit,
    LazyBridge_ChildInit,
    LazyBridge_Request,
    LazyBridge_Finalize,
    LazyBridge_ExitHandler,
    LazyBridge_Interp
};</programlisting>

	<para>
		After the server initialization stage, child processes read the configuration 
		and modules build their own configuration representation. MPM bridges hooks into
		this stage to store and/or build data structures relevant to their design.
		A fundamental information built during this stage is the database of virtual hosts.
		The lazy bridge keeps an array of virtual host descriptor pointers
		each of them referencing an instance of the following structure.
	</para>
	<programlisting>/* virtual host thread queue descriptor */

typedef struct vhost_iface {
    int                 threads_count;      /* total number of running and idle threads */
    apr_thread_mutex_t* mutex;              /* mutex protecting 'array'                 */
    apr_array_header_t* array;              /* LIFO array of lazy_tcl_worker pointers   */
} vhost;</programlisting>

 	<para>
 		A pointer to this data structure array is stored in the bridge status which is a basic
 		structure that every bridge has to create.
	</para>
	<programlisting>/* Lazy bridge internal status data */

typedef struct mpm_bridge_status {
    apr_thread_mutex_t* mutex;
    int                 exit_command;
    int                 exit_command_status;
    int                 server_shutdown;    /* the child process is shutting down  */
    vhost*              vhosts;             /* array of vhost descriptors          */
} mpm_bridge_status;</programlisting>

	<para>
		The lazy bridge also extends the thread private data structure with the
		data concerning the Tcl intepreter 
	</para>

	<programlisting>/* lazy bridge thread private data extension */

typedef struct mpm_bridge_specific {
    rivet_thread_interp*  interp;           /* thread Tcl interpreter object        */
    int                   keep_going;       /* thread loop controlling variable     */
} mpm_bridge_specific;</programlisting>

	<para>
		By design the bridge must create exactly one instance of <command>mpm_bridge_status</command>
		and store its pointer in <command>module_globals->mpm</command>.
		This is usually done at the very beginning of the child init script function pointed by 
		<command>mpm_child_init</command> in the <command>rivet_bridge_table</command> structure.
		For the lazy bridge this field points to <command>Lazy_MPM_ChildInit</command> function
	</para>
	<programlisting>/*
 * -- LazyBridge_ChildInit
 * 
 * child process initialization. This function prepares the process
 * data structures for virtual hosts and threads management
 *
 */

void LazyBridge_ChildInit (apr_pool_t* pool, server_rec* server)
{
    apr_status_t    rv;
    server_rec*     s;
    server_rec*     root_server = module_globals-&gt;server;

    module_globals-&gt;mpm = apr_pcalloc(pool,sizeof(mpm_bridge_status));

    /* This mutex is only used to consistently carry out these 
     * two tasks
     *
     *  - set the exit status of a child process (hopefully will be 
     *    unnecessary when Tcl is able again of calling 
     *    Tcl_DeleteInterp safely) 
     *  - control the server_shutdown flag. Actually this is
     *    not entirely needed because once set this flag 
     *    is never reset to 0
     *
     */

    rv = apr_thread_mutex_create(&amp;module_globals-&gt;mpm-&gt;mutex,
                                  APR_THREAD_MUTEX_UNNESTED,pool);
    ap_assert(rv == APR_SUCCESS);

    /* the mpm-&gt;vhosts array is created with as many entries as the number of
     * configured virtual hosts */

    module_globals-&gt;mpm-&gt;vhosts = 
        (vhost *) apr_pcalloc(pool,module_globals-&gt;vhosts_count*sizeof(vhost));
    ap_assert(module_globals-&gt;mpm-&gt;vhosts != NULL);

    /*
     * Each virtual host descriptor has its own mutex controlling
     * the queue of available threads
     */
     
    for (s = root_server; s != NULL; s = s-&gt;next)
    {
        int                 idx;
        apr_array_header_t* array;
        rivet_server_conf*  rsc = RIVET_SERVER_CONF(s-&gt;module_config);

        idx = rsc-&gt;idx;
        rv  = apr_thread_mutex_create(&amp;module_globals-&gt;mpm-&gt;vhosts[idx].mutex,
                                      APR_THREAD_MUTEX_UNNESTED,pool);
        ap_assert(rv == APR_SUCCESS);
        array = apr_array_make(pool,0,sizeof(void*));
        ap_assert(array != NULL);
        module_globals-&gt;mpm-&gt;vhosts[idx].array = array;
        module_globals-&gt;mpm-&gt;vhosts[idx].threads_count = 0;
    }
    module_globals-&gt;mpm-&gt;server_shutdown = 0;
}</programlisting>
	</section>
	<section>
	 <title>Handling Tcl's exit core command</title> 
	<para>
		Most fields in the <command>mpm_bridge_status</command> are meant to deal
		with the child exit process. Rivet supersedes the Tcl core's exit function
		with <command>::rivet::exit</command> command and it does so in order to curb
        the side effects of the core function Tcl_Exit that would force a whole child process
        to exit immediately. This could have unwanted consequences in any
        environment, like missing the execution of code dedicated to release locks or
        remove files. For threaded MPMs the abrupt child process termination could be
        even more disruptive as all threads will be deleted right away.
	</para>
	<para>
		The <command>::rivet::exit</command> implementation calls the function pointed by
		<command>mpm_exit_handler</command> which is bridge specific. Its main duty
		is to take the proper action in order to release resources and force the
		bridge controlled threads to exit.
	</para>
	<note>
		Nonetheless the <command>exit</command> command should be avoided in ordinary mod_rivet
		programming. We cannot stress this point enough. If your application must bail out
		for some reason focus your attention on the design to find the most appropriate
		route to exit and whenever possible avoid 
		calling <command>exit</command> (which basically wraps a
		C call to Tcl_Exit) at all. Anyway the Rivet implementation partially transforms
		<command>exit</command> in a sort of special <command>::rivet::abort_page</command>
		implementation whose eventual action is to call the <command>Tcl_Exit</command>
		library call. See the <command><xref linkend="exit">::rivet::exit</xref></command>
		command code for further explanations.
	</note>
	<para>
		Both the worker bridge and lazy bridge 
		implementations of <command>mpm_exit_handler</command> call the function pointed 
		by <command>mpm_finalize</command> which is also the function called by the framework
		when the web server shuts down. See these functions' code for further details,
		they are very easy to read and understand
	</para>
	</section>
	<section>
		<title>HTTP request processing with the lazy bridge</title>
	<para>
		Requests processing with the lazy bridge is done by determining for which
		virtual host a request was created. The <command>rivet_server_conf</command>
		structure keeps a numerical index for each virtual host. This index is used
		to reference the virtual host descriptor and from it the request
		handler tries to gain lock on the mutex protecting the array of <command>lazy_tcl_worker</command>
		structure pointers. Each instance of this structure is a descriptor of a thread created for
		a specific virtual host; threads available for processing have their descriptor
		on that array and the handler callback will pop the first
		<command>lazy_tcl_worker</command> pointer to signal the thread
		there is work to do for it. This is the <command>lazy_tcl_worker</command> structure
	</para>
	<programlisting>/* lazy bridge Tcl thread status and communication variables */

typedef struct lazy_tcl_worker {
    apr_thread_mutex_t* mutex;
    apr_thread_cond_t*  condition;
    int                 status;
    apr_thread_t*       thread_id;
    server_rec*         server;
    request_rec*        r;
    int                 ctype;
    int                 ap_sts;
    rivet_server_conf*  conf;               /* rivet_server_conf* record */
} lazy_tcl_worker;</programlisting>
	<para>
		The server field is initialized with the virtual host server record. Whereas the <command>conf</command>
		field keeps the pointer to a run time computed <command>rivet_server_conf</command>. This structure
		may change from request to request because the request configuration changes when the URL may refer 
		to directory specific configuration in <command>&lt;Directory ...&gt;...&lt;/Directory&gt;</command> 
		blocks
	</para>
	<para>
		The Lazy bridge will not start any Tcl worker thread at server startup, but it will
		wait for requests to come in and then if worker threads are sitting on a virtual host queue
		a thread's <command>lazy_tcl_worker</command> structure pointer is popped
		and the request handed to it. If no available thread is on the queue a new worker thread is 
		created. The code in the <command>Lazy_MPM_Request</command> easy to understand and shows
		how this is working
	</para>
	<programlisting>/* -- LazyBridge_Request
 *
 * The lazy bridge HTTP request function. This function
 * stores the request_rec pointer into the lazy_tcl_worker
 * structure which is used to communicate with a worker thread.
 * Then the array of idle threads is checked and if empty
 * a new thread is created by calling create_worker
 */

int LazyBridge_Request (request_rec* r,rivet_req_ctype ctype)
{
    lazy_tcl_worker*    w;
    int                 ap_sts;
    rivet_server_conf*  conf = RIVET_SERVER_CONF(r->server->module_config);
    apr_array_header_t* array;
    apr_thread_mutex_t* mutex;

    mutex = module_globals->mpm->vhosts[conf->idx].mutex;
    array = module_globals->mpm->vhosts[conf->idx].array;
    apr_thread_mutex_lock(mutex);

    /* This request may have come while the child process was
     * shutting down. We cannot run the risk that incoming requests
     * may hang the child process by keeping its threads busy,
     * so we simply return an HTTP_INTERNAL_SERVER_ERROR.
     * This is hideous and explains why the 'exit' commands must
     * be avoided at any costs when programming with mod_rivet
     */

    if (module_globals->mpm->server_shutdown == 1)
    {
        ap_log_rerror(APLOG_MARK,APLOG_ERR,APR_EGENERAL,r,
                      MODNAME ": http request aborted during child process shutdown");
        apr_thread_mutex_unlock(mutex);
        return HTTP_INTERNAL_SERVER_ERROR;
    }

    /* If the array is empty we create a new worker thread */

    if (apr_is_empty_array(array))
    {
        w = create_worker(module_globals->pool,r->server);
    }
    else
    {
        w = *(lazy_tcl_worker**) apr_array_pop(array);
    }

    apr_thread_mutex_unlock(mutex);

    /* Locking the thread descriptor structure mutex */

    apr_thread_mutex_lock(w->mutex);
    w->r        = r;
    w->ctype    = ctype;
    w->status   = init;
    w->conf     = conf;
    apr_thread_cond_signal(w->condition);

    /* we wait for the Tcl worker thread to finish its job */

    while (w->status != done) {
        apr_thread_cond_wait(w->condition,w->mutex);
    }
    ap_sts = w->ap_sts;

    w->status = idle;
    w->r      = NULL;
    apr_thread_cond_signal(w->condition);
    apr_thread_mutex_unlock(w->mutex);

    return ap_sts;
}</programlisting>	
	<para>
		After a request is processed the worker thread returns its own
		lazy_tcl_worker descriptor to the array and then waits
		on the condition variable used to control and synchronize the bridge 
		threads with the Apache worker threads. The worker thread code
		is the request_processor function
	</para>
	<programlisting>/*
 * -- request_processor
 *
 * The lazy bridge worker thread. This thread initializes its control data and
 * prepares to serve requests addressed to the virtual host which is meant to work
 * as a content generator. Virtual host server data are stored in the
 * lazy_tcl_worker structure stored in the generic pointer argument 'data'
 *
 */

static void* APR_THREAD_FUNC request_processor (apr_thread_t *thd, void *data)
{
    lazy_tcl_worker*        w = (lazy_tcl_worker*) data;
    rivet_thread_private*   private;
    int                     idx;
    rivet_server_conf*      rsc;

    /* The server configuration */

    rsc = RIVET_SERVER_CONF(w->gt;server->gt;module_config);

    /* Rivet_ExecutionThreadInit creates and returns the thread private data. */

    private = Rivet_ExecutionThreadInit();

    /* A bridge creates and stores in private->gt;ext its own thread private
     * data. The lazy bridge is no exception. We just need a flag controlling
     * the execution and an intepreter control structure */

    private->gt;ext = apr_pcalloc(private->gt;pool,sizeof(mpm_bridge_specific));
    private->gt;ext->gt;keep_going = true;
    RIVET_POKE_INTERP(private,rsc,Rivet_NewVHostInterp(private->gt;pool,rsc->gt;default_cache_size));
    private->gt;ext->gt;interp->gt;channel = private->gt;channel;

    /* The worker thread can respond to a single request at a time therefore
       must handle and register its own Rivet channel */

    Tcl_RegisterChannel(private->gt;ext->gt;interp->gt;interp,*private->gt;channel);

    /* From the rivet_server_conf structure we determine what scripts we
     * are using to serve requests */

    private->gt;ext->gt;interp->gt;scripts =
            Rivet_RunningScripts (private->gt;pool,private->gt;ext->gt;interp->gt;scripts,rsc);

    /* This is the standard Tcl interpreter initialization */

    Rivet_PerInterpInit(private->gt;ext->gt;interp,private,w->gt;server,private->gt;pool);

    /* The child initialization is fired. Beware the terminological
     * trap: we inherited from fork capable systems the term 'child'
     * meaning 'child process'. In this case the child init actually
     * is a worker thread initialization, because in a threaded module
     * this is the agent playing the same role a child process plays
     * with the prefork bridge */

    Lazy_RunConfScript(private,w,child_init);

    idx = w->gt;conf->gt;idx;

    /* After the thread has run the configuration script we
       increment the threads counter */

    apr_thread_mutex_lock(module_globals->gt;mpm->gt;vhosts[idx].mutex);
    (module_globals->gt;mpm->gt;vhosts[idx].threads_count)++;
    apr_thread_mutex_unlock(module_globals->gt;mpm->gt;vhosts[idx].mutex);

    /* The thread is now set up to serve request within the the
     * do...while loop controlled by private->gt;keep_going  */

    apr_thread_mutex_lock(w->gt;mutex);
    do
    {
        while ((w->gt;status != init) &amp;&amp; (w->gt;status != thread_exit)) {
            apr_thread_cond_wait(w->gt;condition,w->gt;mutex);
        }
        if (w->gt;status == thread_exit) {
            private->gt;ext->gt;keep_going = false;
            continue;
        }

        w->gt;status = processing;

        /* Content generation */

        private->gt;req_cnt++;
        private->gt;ctype = w->gt;ctype;
        private->gt;r = w->gt;r;

        w->gt;ap_sts = Rivet_SendContent(private);

        w->gt;status = done;
        apr_thread_cond_signal(w->gt;condition);
        while (w->gt;status == done) {
            apr_thread_cond_wait(w->gt;condition,w->gt;mutex);
        }

        /* rescheduling itself in the array of idle threads */

        apr_thread_mutex_lock(module_globals->gt;mpm->gt;vhosts[idx].mutex);
        *(lazy_tcl_worker **) apr_array_push(module_globals->gt;mpm->gt;vhosts[idx].array) = w;
        apr_thread_mutex_unlock(module_globals->gt;mpm->gt;vhosts[idx].mutex);

    } while (private->gt;ext->gt;keep_going);
    apr_thread_mutex_unlock(w->gt;mutex);

    Lazy_RunConfScript(private,w,child_exit);
    ap_log_error(APLOG_MARK,APLOG_DEBUG,APR_SUCCESS,w->gt;server,"processor thread orderly exit");

    apr_thread_mutex_lock(module_globals->gt;mpm->gt;vhosts[idx].mutex);
    (module_globals->gt;mpm->gt;vhosts[idx].threads_count)--;
    apr_thread_mutex_unlock(module_globals->gt;mpm->gt;vhosts[idx].mutex);

    apr_thread_exit(thd,APR_SUCCESS);
    return NULL;
}</programlisting>
	<para>
		The lazy bridge <command>module_globals->bridge_jump_table->mpm_thread_interp</command>, which
		is supposed to return the rivet_thread_interp structure pointer relevant to a given
		request, has a straightforward task to do since by design each thread has
		one interpreter
	</para>
	<programlisting>rivet_thread_interp* LazyBridge_Interp (rivet_thread_private* private,
                                      rivet_server_conf*    conf,
                                      rivet_thread_interp*  interp)
{
    if (interp != NULL) { private->ext->interp = interp; }

    return private->ext->interp;
}
</programlisting>
	<para>
		As already pointed out
		running this bridge you get separate virtual interpreters and separate channels by default
		and since by design each threads gets its own Tcl interpreter and Rivet channel you will
		not be able to revert this behavior in the configuration with 
	</para>
	<programlisting>SeparateVirtualInterps Off
SeparateChannels       Off</programlisting>
	<para>
		which are simply ignored
	</para>
	</section>
 </section>
