DataTables - Passing Data to the Server in Client-Side processing mode

DataTables is AWESOME. I use it for list-based selection, table-based editing, data display and filtering and anything else I can think of because it is so robust that virtually anything is possible. Really.

If the dataset being used is fairly small (you may interpret small any way you like), DataTables client-side processing is amazing. It sorts and filters, updates all the navigation, lets you adjust the number of records displayed - as well as many more optional features - with very little effort.

Client-side processing - where filtering, paging and sorting calculations are all performed in the web-browser.


Ref: https://datatables.net/manual/data/#Client-side-processing

but

For client-side processing no additional data is submitted to the server

Ref: https://datatables.net/reference/option/ajax.data

The goal was to have a single client-side datatable which would display different content based on the user's actions. For example, 'list all the items with a status of "new" and a (time) segment of 1'.

HTML for the DataTable

<table id="datatable" class="display" cellspacing="0" width="100%">
  <thead>
    <tr>
      <th>Id</th>
      <th>Vendor</th>
      <th>Item</th>
      <th>Date</th>
    </tr>
  </thead>
</table>

JavaScript


var datatable;
$(function($){
    datatable = $('#datatable').DataTable(
         {
           "ajax": {
                        "url":  "data-10-1.php"
           },
           "columns":
           [
               {"data": "link"},
               {"data": "vendor"},
               {"data": "item"},
               {"data": "date"}
          ]
  });
  // Handle the click events
  $(".summary").on("click", function(evt){
        var target = evt.target, status, segment;
        if (target.hasAttribute("data-status")) {
            status=$(target).attr("data-status");
            segment=$(target).attr("data-segment");
            datatable.ajax.url("data-"+status+"-"+segment+".php");
            datatable.ajax.reload();
            datatable.draw('full-reset');
        }
    });
});

Notice that the URLs are variable. The status and segment values are passed in the URL itself, there is no POST or GET data.

Sample HTML which invokes the event handler. This HTML is inside the div with class="summary".

 <span class="details" data-status="10" data-segment="4" data-status-text="new">84</span>

In order to extract the status and segment out of the URL, you can use Apache RewriteRules, like so:


<directory /var/www/html>
    RewriteEngine On
    RewriteRule  ^(data)-(\d+)-(\d+)(\.php)$ $1$4?status=$2&segment=$3 [L]
</directory>

status and segment can then be accessed as $_GET variables. Be sure to validate them.

How does it work? As the user clicks on the spans, the event handler reads the data attributes and constructs a URL that is sent to the server. When Apache receives the request, it rewrites it to deliver the data embedded in the URL as GET parameters.

This post courtesy of Game Creek Video

Simulate a cron job with watch

If you want to test a script that will run as a cron job, or you would like to run a command line script repeatedly to test it, watch is a great tool.

watch -n [number of seconds between runs] [command to execute]

For example:

watch -n 60 php super-script.php

Web application session timeout code

Session timeout warning for a web application with a variety of page layouts and frequent use of multiple tabs.

Nutshell explanation - ping the server, if you get a 403, show a huge red bar across the top of whatever page pinged.

Result:

  • It is immediately apparent the tab has timed out
  • If one tab times out, it does not disrupt the others
  • There is a link to help the user log in again

function sessionPing() {
    var pinger;
    function doPing() {
        var pReq = new XMLHttpRequest();
        pReq.addEventListener("load", function () {
            var sessionExpired, sessionExpiredMessage;
            var reloadLink;
            if (this.status === 403) {
                clearInterval(pinger);
                sessionExpired = document.createElement("div");
                // Sometimes an inline style is really the best solution
                sessionExpired.setAttribute("style","display:block; width: 100%; line-height: 2.5em; position:absolute; top:0; z-index:10000; text-align: center; background-color: #f00; color: #fff; font-family: 'Trebuchet MS',sans; font-size: 1.5em; font-style: italic");
                sessionExpiredMessage = document.createTextNode("Session expired ");
                sessionExpired.appendChild(sessionExpiredMessage);
                reloadLink = document.createElement("a");
                reloadLink.href = location.href;
                reloadLink.textContent = "Click to Continue";
                reloadLink.setAttribute("style","font-size:0.7em;color:#ddd");
                sessionExpired.appendChild(reloadLink);
                document.body.insertBefore(sessionExpired, document.body.firstChild);
                document.title = "Session expired";
            }
        });
        pReq.open("GET", "/ping.php");
        pReq.send();
    }
    pinger = setInterval(doPing, 30000);
}
sessionPing();


<?php
session_start();
if (empty($_SESSION['user_id'])) {
    header('HTTP/1.1 403 Forbidden');
    exit;
}
header('HTTP/1.1 201 No content');
exit;

One may argue that the page should be cleared, in this case, I chose to leave it up so people can copy the content off.

Confession: I didn't test this code, it is an extract.

This post courtesy of Game Creek Video

Symfony 3 Ajax (EntityType) CollectionType

Objective: Create an efficient way to use a Symfony CollectionType of Entities. I wanted to be able to provide a Dijit FilteringSelect (autocomplete) for a collection of entities.

Issues: Using a CollectionType of EntityTypes loaded all the entities into a selection statement. This is fine if there are only a few entities, but not if there are hundreds. Also, model transformers cannot be used with CollectionTypes.

Solution: Create a custom FieldType and apply a model transformer.

The custom field type is based off a TextType, which allows the id of the entity to be passed as text, then transformed by the transformer. Transformers are described here.

The Dijit FilteringSelect only needs an id to serve as an anchor within the page layout. One may argue it could all be dynamic, but I like to use the HTML as a foundation. A text input is used as the base element which is replaced by the Dijit widget. The widget is created and populated with the data from a store.

<?php

namespace AppBundle\Form\Admin\Asset\Type;

use Symfony\Component\Form\AbstractType;
use Symfony\Component\Form\FormBuilderInterface;
use Symfony\Component\OptionsResolver\OptionsResolver;
use Symfony\Component\Form\Extension\Core\Type\TextType;
use AppBundle\Form\Admin\Asset\DataTransformer\ModelToIdTransformer;

class ModelRelationshipType extends AbstractType
{

    private $modelToIdTransformer;

    public function __construct( ModelToIdTransformer $modelToIdTransformer )
    {
        $this->modelToIdTransformer = $modelToIdTransformer;
    }

    /**
     * @param FormBuilderInterface $builder
     * @param array $options
     */
    public function buildForm( FormBuilderInterface $builder, array $options )
    {

        $builder
                ->addModelTransformer( $this->modelToIdTransformer );
    }

    public function configureOptions( OptionsResolver $resolver )
    {
        // Stub
    }

    public function getParent()
    {
        return TextType::class;
    }

}

The new custom field type was then added as a service, with the transformer injected in:

    app.admin.field.type.model_relationship:
        class: AppBundle\Form\Admin\Asset\Type\ModelRelationshipType
        arguments: ['@app.form.data_transformer.model']
        tags:
            - { name: form.type, alias: 'app_model_relationship_type' }

The Twig template code to display each row of the collection uses a macro, because there are four of these fields.

{% block _model_extends_entry_row %}
{% import 'form/macros.html.twig' as form_macros %}
{{ form_macros.model_relationship_entry_row('requires',form) }}
{% endblock %}

{%macro relation(title,type,form) %}{%spaceless%}
        <div id="{{type}}" class="{{type}}">
            <h3>{{title}}</h3>
            {{form_row(form)}}
            {% if form.vars.allow_add %}
                <div data-type="{{type}}" class="add-one-more-row">{{ 'common.add_one_more'|trans}}
            {% endif %}
        </div>
{%endspaceless%}{%endmacro %}

To use the new field type in the form, you add the CollectionType, and the entry_type is ModelRelationshipType.

->add( 'requires', CollectionType::class, [
                    'entry_type' => ModelRelationshipType::class,
                    'required' => false,
                    'label' => false,
                    'empty_data' => null,
                    'allow_add' => true,
                    'allow_delete' => true,
                    'delete_empty' => true
                ] )

KnpMenu - JSON Renderer to support Dojo Dijit/Tree - Ajax

This is a modified version of the earlier post which created a static JSON representation of a menu.

What makes it different is that the earlier version rendered the items from the MenuBuilder into a static JSON object that was embedded in a page with a script tag. This approach removes the script tag (and the unnecessary twig template elements) and switches to a more graceful and efficient lazy loading menu.

The menu renderer is organizing the menu into the JSON format used by a Dijit Tree. This is set up for lazy loading, so it it only delivering the items in the assets tree. Items with URLs are pages, which you may click on to view the page, or expand to load more of the menu. These elements are defined in the menu builder (which isn't included in this post).

Dijit/Tree JSON

{
  "id": "admin-assets",
  "name": "Assets",
  "uri": null,
  "has_children": true,
  "parent": "admin",
  "children": [
    {
      "id": "equipment",
      "name": "Equipment",
      "uri": "\/admin\/asset\/equipment",
      "has_children": false,
      "parent": "admin-assets"
    },
    {
      "id": "issues",
      "name": "Issues",
      "uri": "\/admin\/issue\/issue",
      "has_children": false,
      "parent": "admin-assets"
    },
    {
      "id": "manufacturers",
      "name": "Manufacturers",
      "uri": "\/admin\/asset\/manufacturer",
      "has_children": true,
      "parent": "admin-assets"
    },
    {
      "id": "trailers",
      "name": "Trailers",
      "uri": "\/admin\/asset\/trailer",
      "has_children": true,
      "parent": "admin-assets"
    },
    {
      "id": "vendors",
      "name": "Vendors",
      "uri": "\/admin\/asset\/vendor",
      "has_children": true,
      "parent": "admin-assets"
    },
    {
      "id": "configuration",
      "name": "Configuration",
      "uri": null,
      "has_children": true,
      "parent": "admin-assets"
    }
  ]
}

Content Menu Options

The content menu options allow the user to navigate directly to content they have created in the system. In this case, it is manufacturers. As above, if a URL is provided, the user may go to the page indicated by the uri attribute, or expand the menu item to view the children. The children under a manufacturer menu item are brands.

The URL is /api/menustore/adminmenus/manufacturers.

{
  "children": [
    {
      "id": "manufacturer-14",
      "name": "Pelican",
      "parent": "manufacturer",
      "has_children": "1",
      "uri": "\/admin\/asset\/manufacturer?name=Pelican"
    },
    {
      "id": "manufacturer-2",
      "name": "Shed",
      "parent": "manufacturer",
      "has_children": "0",
      "uri": "\/admin\/asset\/manufacturer?name=Shed"
    },
    {
      "id": "manufacturer-1",
      "name": "Barn",
      "parent": "manufacturer",
      "has_children": "1",
      "uri": "\/admin\/asset\/manufacturer?name=Barn"
    }
  ]
}

Lazy loaded element

The content menu for the manufacturer named Barn follows. This leaf of the tree doesn't have any children. Clicking on it will take the user to the URL in the uri attribute.

The URL is /api/menustore/adminmenus/manufacturer-1

{
  "children": [
    {
      "id": 12,
      "name": "Barn",
      "parent": "manufacturer-1",
      "uri": "\/admin\/asset\/manufacturer\/Barn\/brand\/Barn",
      "has_children": false,
      "children": null
    }
  ]
}

This can be read as the manufacturer's name is Barn and they have one brand, Barn. If you click on the brand Barn under the manufacturer Barn, you will go to a page to manage the brand Barn.

The render code which returns the menu rendered ready for a Dijit/Tree. Reference.

    public function render( ItemInterface $item, array $options = array() )
    {
        $options = array_merge( $this->defaultOptions, $options );
        
        $translator = $options['translator'];

        $itemIterator = new \Knp\Menu\Iterator\RecursiveItemIterator( $item );

        $iterator = new \RecursiveIteratorIterator( $itemIterator, \RecursiveIteratorIterator::SELF_FIRST );

        $items = [];
        foreach( $iterator as $item )
        {
            $translatedLabel = $translator->trans($item->getLabel());
            $id = $item->getName();  
            $itemData = [ 'id' => strtolower( $item->getName() ), 'name' => $translatedLabel, 'uri' => $item->getUri()];
            $itemData['has_children'] = $item->hasChildren();
            $parentId = $item->getParent()->getName();
            if ($parentId !== $id) {
                $itemData['parent'] = strtolower($parentId);
                if (!isset($items[$parentId]['children'])) {
                    $items[$parentId]['children'] = [];
                }
                $items[$parentId]['children'][] = $itemData;
            }
            if (isset($items[$id])) {
                $items[$id] = array_merge($itemData, $items[$id]);
            } else {
                $items[$id] = $itemData;
            }
        }
        return $items;
    }

The menu builder and renderer are defined as services.

    app.menu_builder:
        class: AppBundle\Menu\MenuBuilder
        arguments: ["@knp_menu.factory"]
        calls:
            - [setContainer, ["@service_container"]]
        tags:
            - { name: knp_menu.menu_builder, method: createAdminMenu, alias: admin }
            - { name: knp_menu.menu_builder, method: createCalendarMenu, alias: calendar }
            - { name: knp_menu.menu_builder, method: createMainMenu, alias: main }
            - { name: knp_menu.menu_builder, method: createUserMenu, alias: user }
            - { name: knp_menu.menu_builder, method: createVendorMenu, alias: vendor }

    app.menu_renderer:
        # The class implements Knp\Menu\Renderer\RendererInterface
        class: AppBundle\Menu\JsonRenderer
        arguments: [ "@twig",  "knp_menu.html.twig", "@knp_menu.matcher", {"translator": "@translator" }]
        tags:
            # The alias is what is used to retrieve the menu
            - { name: knp_menu.renderer, alias: json }

The static routing is as follows (this is output from

php bin/console debug:route

r) :


app_admin_api_menu_store_get_adminmenu GET ANY ANY /api/menustore/adminmenus
app_admin_api_menu_store_get_adminmenu_alt GET ANY ANY /api/menustore/adminmenus/
app_admin_api_menu_store_get_adminmenu_parent GET ANY ANY /api/menustore/adminmenus/?parent={parent}
app_admin_api_menu_store_get_adminmenu_id GET ANY ANY /api/menustore/adminmenus/{id}

The code which delivers the admin menu follows.

This code is delivering the static menu elements, those defined in the MenuBuilder, using the JsonRenderer listed above.

They provide references which allow the menu to lazy load additional content.

The JSON listed above is from, /api/menustore/adminmenus/admin-assets.

Menu elements for content, are listed first by the name of the content type (trailers) and allow loading of the contents of the

app_admin_api_store_get_trailer GET ANY ANY /api/store/trailer
app_admin_api_store_get_trailercontents GET ANY ANY /api/store/trailercontents/{id}

    /**
     * @View()
     * @Get("/adminmenus")
     * @Get("/adminmenus/", name="_alt")
     * @Get("/adminmenus/?parent={parent}", name="_parent", defaults={"parent" = "admin"})
     * @Get("/adminmenus/{id}", name="_id", defaults={"id" = "admin"})
     *
     */
    public function getAdminmenuAction( Request $request )
    {
        $this->denyAccessUnlessGranted( 'ROLE_ADMIN', null, 'Unable to access this page!' );
        $adminMenu = $this->get( 'app.menu_builder' )->createAdminMenu( [] );
        $renderer = $this->get( 'app.menu_renderer' );
        $menu = [];
        $id = $request->get( 'id' );
        if( $id !== null )
        {
            // This renders URLs without an id
            if( !in_array( $id, $this->entityMenus ) && preg_match( '/\-\d+$/', $id ) === 0 )
            {
                $menu = $renderer->render( $adminMenu, ['depth' => 1] );
                $menu = $menu[$id];
            }
            else
            {
                // This renders dynamic menu elements - the dynamicId would be manufacturer for this blog post
                $dynamicId = trim( ucfirst( preg_replace( '/^([a-z]+).*$/', '$1', $id ) ), 's' );
                $menuMethod = 'get' . $dynamicId . 'Menu';
                if( method_exists( $this, $menuMethod ) )
                {
                    $menu = $this->{$menuMethod}( $adminMenu, $renderer, $id );
                }
            }
        }
        $parent = $request->get( 'parent' );
        if( $parent !== null )
        {
            foreach( $adminMenu as $name => $children )
            {
                if( $name === $parent )
                {
                    $menu['id'] = $parent;
                    $menu['name'] = $name;
                    $menu['children'] = $renderer->render( $children, ['depth' => 1], 'json' );

                    break;
                }
            }
        }
        if( isset( $menu['children'] ) )
        {
            foreach( $menu['children'] as $c => $child )
            {
                if( in_array( $child['id'], $this->entityMenus ) )
                {
                    $menu['children'][$c]['has_children'] = true;
                }
            }
        }
        return $menu;
    }

This code handles the lazy loading. The id may be manufacturers or manufacturer-id. If it is the former, a list of the manufacturers will be returned, the later will return all the brands for a manufacturer. If the system is likely to have a large number of manufacturers or sub items, you should add another layer to organize the elements by the first letter of their name - so you can click on A to see all the manufacturers whose name starts with A.

    function getManufacturerMenu( $adminMenu, $renderer, $id )
    {
        $limit = 2500; // TODO: Change to deliver first letters if there are too many manufacturers
        $em = $this->getDoctrine()->getManager();
        $base = explode( '-', $id );
        switch( $base[0] )
        {
            case 'manufacturers':
                $queryBuilder = $em->createQueryBuilder();
                $queryBuilder
                        ->select( ["CONCAT('manufacturer-',m.id) AS id", 'm.name', "'manufacturer' AS parent", 'COUNT(b.id) AS has_children'] )
                        ->from( 'AppBundle\Entity\Asset\Manufacturer', 'm' )
                        ->leftJoin( 'm.brands', 'b' )
                        ->orderBy( 'm.name' )
                        ->groupBy( 'm.id' )
                        ->setFirstResult( 0 )
                        ->setMaxResults( $limit );
                $menu = $renderer->render( $adminMenu['admin']['admin-assets']['manufacturers'] );
                $children = $queryBuilder->getQuery()->getResult();
                $l = count( $children );
                if( $l < $limit )
                {
                    for( $i = 0; $i < $l; $i++ )
                    {
                        $children[$i]['uri'] = $this->generateUrl(
                                'app_admin_asset_manufacturer_index', ['name' => $children[$i]['name']], true ); // absolute
                    }
                }
                $menu['children'] = $children;
                break;
            case 'manufacturer':
                $manufacturer = $em->getRepository( 'AppBundle\Entity\Asset\Manufacturer' )->find( $base[1] );
                $brands = $manufacturer->getBrands();
                $children = [];
                foreach( $brands as $b )
                {
                    $children[] = [
                        'id' => $b->getId(),
                        'name' => $b->getName(),
                        'parent' => $id,
                        'uri' => $this->generateUrl(
                                'app_admin_asset_manufacturer_getmanufacturerbrand', ['mname' => $manufacturer->getName(), 'bname' => $b->getName()], true ), // absolute
                        'has_children' => false,
                        'children' => null];
                }
                $menu['children'] = $children;
                break;
        }
        return $menu;
    }

The (Dojo) client side code:


define([
    "dojo/dom",
    "dojo/store/Memory",
    "dijit/tree/ObjectStoreModel",
    "dojo/store/JsonRest",
    "dijit/Tree",
    "dojo/domReady!"
], function (dom,
        Memory, ObjectStoreModel, JsonRest, Tree) {
//"use strict";
    function run() {

        var store = new JsonRest({
            target: "/api/menustore/adminmenus/",
            getChildren: function (object) {
                return this.get(object.id).then(function (fullObject) {
                    return fullObject.children;
                });
            }
        });

        var model = new ObjectStoreModel({
            store: store,
            mayHaveChildren: function (object) {
                return object.has_children;
            }, getRoot: function (onItem) {
                this.store.get("admin").then(onItem);
            },
        });
        // Create the Tree.
        var tree = new Tree({
            id: "admin-menu",
            model: model,
            persist: true,
            onClick: function (item) {
                if( typeof item.uri !== "undefined" && item.uri !== null ) {
                    location.href = item.uri;
                }
            },
            getIconClass: function (item, opened) {
                return (item && item.has_children) ? (opened ? "dijitFolderOpened" : "dijitFolderClosed") : "dijitLeaf"
            }
        });
        tree.placeAt(dom.byId("admin-left-menu"));
        tree.startup();
    }
    return {
        run: run
    };
});
//# sourceURL=menu.js