Chapter 20. Extending Rax

Rax provides a mechanism for easy extending its functionality: external functions and procedures. A function or procedure in Rax, can be declared as external using the following syntax:

<function_type> : <function_name> <- extern [ ( <external_function_name> [, <path_to_library> ) ]]

The external function name and the path to the library are optional. If the external function name is not provided, it is assumed to be the same as the function name. The path to the library should contain a full path to the library containing the implementation of the function, including the name of the library file, but excluding its extension. This is to preserve the portability of Rax scripts between different platforms. The file extension will be namely .dll on Windows, .so on Linux and FreeBSD and .dylib on OSX. To specify the path to the file, you can use the __EXE_PATH__ macro. If the path to the library is not provided, Rax will search in its current working directory, and it will assume that the library file name is the same as the name of the current script its executing, except for the file extension, and with a "rx_" prefix. For example:

    // PluginTest.rax
    //
    [~,~,@,@,#,#,$,$,^,^,&,&,[#],[#],{#},{#}]: tupswap_arg;

    \tupswap_arg <- \tupswap_arg: tupswap  <- extern;

    \tupswap_arg <- \tupswap_arg: tupswap1 <- extern("tupswap");

    \tupswap_arg <- \tupswap_arg: tupswap2 <- extern("tupswap",
                                                     __EXE_PATH__
                                                     "rx_PluginTest");
     

The above example illustrates three ways of declaring an external function. All three functions, tupswap, tupswap1 and tupswap2 refer to the same external function "tupswap" implemented in the rx_PluginTest.dll library. The rx_PluginTest.dll is located in the same directory as the Rax executable (__EXE_PATH__).

Similarly, output procedures can be implemented externally. For example, the graphical library in Rax is implemented as a set of external output procedures. The example below, is a declaration of one of the graphical procedures from the GraphicalEngine.rax file:

    ` <- [&*2] : `lineto <- extern("Plineto",
                                  __EXE_PATH__
                                  "rx_GraphicalEngine/rx_GraphicalEngine");
       

Rax provides a C API that allows easy manipulating of Rax types and data structures. To have access to this API, rax.h and rax_extern.h need to be included. rax.h contains the definitions of basic types used in Rax and rax_extern.h contains the signatures of the Rax API functions. Let us first take a look at the implementation of a simple scalar function "logistic":

    // rx_Logistic.c
    // author: jms
    // The implementation of the logistic function
    //
    RAX_EXTERNAL_FUNCTION
    rx_logistic(u_rax_value_t *p_out, const u_rax_value_t in,
                rax_type_t type_out, rax_type_t type_in,
                void *rax_api_ftab[])
    {
      u_rax_value_t *av = rax_value_tuple_array_ref(in);

      p_out->rax_real= av[0].rax_real*(-1 + 2/(1 + exp(-av[1].rax_real * av[2].rax_real)));

      return NULL; /* No errors to report. */
    }
       

The first thing to notice in this example is the external function signature, which always has the form:

    RAX_EXTERNAL_FUNCTION
    rx_[funcname](u_rax_value_t *p_out, const u_rax_value_t in,
                  rax_type_t type_out, rax_type_t type_in,
                  void *rax_api_ftab[])
       

where '[funcname]' is the name of the external function. The in parameter describes the function's input value. This value is of type u_rax_value_t which is defined in rax.h and is a union that can store Rax values of all types. You can extract values of various types from this union in the following way:

    u_rax_value_t v;
    rax_number_t n = v.rax_number;
    rax_real_t   r = v.rax_real;
    rax_string_t s = v.rax_string;
    rax_oset_t   o = v.rax_oset;
    // etc.
       

In the above example, the input parameter is of type tuple. Rax API provides a function to extract tuple fields from a tuple:

u_rax_value_t *rax_value_tuple_array_ref(v_tuple); 
u_rax_value_t v_tuple;
 

This function returns an array of rax values (of type u_rax_value_t), representing values in the tuple fields.

The p_out parameter is a pointer to the variable, in which you have to store the output value of the function. In this case, the return value is a simple real, so we can store it in a simple way:

  p_out->rax_real = ...
       

In more complex scenarios, we might need to build a value of a composite type (tuple or set). Below an example of the 'tupswap' function, that returns a tuple:

    // rx_PluginTest.c
    // author: jms
    // The implementation of the external function 'tupswap'
    //
    #include "rax.h"
    #include "rax_extern.h"

    /* Swap all pairs in a tuple.
    // in:  [~,~,@,@,#,#,$,$,^,^,&,&,[#],[#],{#},{#}]
    // out: [~,~,@,@,#,#,$,$,^,^,&,&,[#],[#],{#},{#}]
    */
    RAX_EXTERNAL_FUNCTION
    rx_tupswap(u_rax_value_t *p_out, const u_rax_value_t in,
               rax_type_t type_out, rax_type_t type_in,
               void *rax_api_ftab[])
    {
       u_rax_value_t *p_in_values;

       if (0 != rax_type_cmp(type_out, type_in))
         return "The input and output types should be the same";

       p_in_values = rax_value_tuple_array_ref(in);

       *p_out = rax_value_tuple_new_vararg(type_out,
                                           p_in_values[1],  p_in_values[0],
                                           p_in_values[3],  p_in_values[2],
                                           p_in_values[5],  p_in_values[4],
                                           p_in_values[7],  p_in_values[6],
                                           p_in_values[9],  p_in_values[8],
                                           p_in_values[11], p_in_values[10],
                                           p_in_values[13], p_in_values[12],
                                           p_in_values[15], p_in_values[14]
                                           );

      return NULL; /* No errors to report. */
    }
       

This function swaps pairs of values in a tuple. To create the output tuple, another Rax API function is used:

u_rax_value_t rax_value_type_tuple_new_vararg(type_out,  
 ...); 
rax_type_t type_out;
 

Rax external function should return NULL on success, and a string containing the error message on error. In the example above, we return an error, if the input and the output types are not equal. We compare the types using yet another Rax API function - rax_type_cmp.

It is also possible to process whole sets in an external function. For this purpose, Rax provides an set enumerator. In the example below, an set enumerator is used to generate a barplot from a dataset:

    /* An implementation of a `barplot output procedure
     * ` <- {[$:seriesLabel, $:xLabel, &:y]}: `barplot;
     */
    RAX_EXTERNAL_FUNCTION
    rx_Pbarplot(u_rax_value_t *p_out, const u_rax_value_t in,
                rax_type_t type_out, rax_type_t type_in,
                void *rax_api_ftab[])
    {
      void *enumerator;
      u_rax_value_t v, *av;
      rax_type_t el_type;
      rax_string_t seriesLabel, xLabel;
      rax_real_t y;

      el_type = rax_type_oset_element_type_ref(type_in);

      /* Create a set enumerator */
      enumerator = rax_value_oset_get_enum_sorted(in, type_in);
      /* Step through set elements */
      while (rax_value_oset_next_value_new(&enumerator, &v)) {
        av = rax_value_tuple_array_ref(v);
        seriesLabel = av[0].rax_string;
        xLabel = av[1].rax_string;
        y = av[2].rax_real;

        /* Do something with the retrieved element
         *
         * ....
        */

        /* Drop the current element before retrieving the next one */
        rax_value_drop(&v, el_type);
      }
      /* Clean up the enumerator */
      rax_value_oset_drop_enum(&enumerator);

       

The function rax_value_oset_get_enum_sorted creates a new set enumerator. The elements will be returned in the order that belongs to this set. There is also a rax_value_oset_get_enum_unsorted which will create an enumerator returning set elements in an unspecified order. The second function can be used for better performance when the order of elements doesn't matter. The function rax_value_oset_next_value_new is used to step through the elements of the set. It creates a new u_rax_value_t value for each element which needs to be dropped before retrieving the next element. Otherwise your external function will leak memory. The rax_value_drop function is used for that purpose. Finally, the rax_value_oset_drop_enum function is used to free the enumerator's internal data structures.

Note that using a set enumerator will cause fetching all elements of a set from the SQL back end to memory for processing. The elements will be processed sequentially, which means that they don't necessarily need to fit all in memory, but still fetching all elements from the underlying SQL database is a costly operation. Do not do it on very large datasets.

A detailed documentation of all Rax API functions can be found in Appendix C, Rax external functions API