web notes - a blog

I have been looking at performance for a web application. I am using ab to compare the response times while adjusting the database indexing.

# 'log in'
curl -L https://www.example.com/login.php -k -b cookie -c cookie-jar -d username=admin -d password=**** -s > /dev/null
# Run the test -n times (in this case 10)
ab -C `grep PHP_session cookie-jar | cut -f6,7 | tr '\t' '='` -n 10 'https://www.example.com/main.php'

What's happening?

The curl command is logging in to the application and saving the cookies in a file named cookie-jar.

The grep command is extracting the session id cookie - name and value - from the cookie-jar and using it to authorize access to a main page for ab.

This post courtesy of Game Creek Video

Have you ever had someone say, "This page is great! Can I get the data and use it in Excel?"

Sometimes, the beautiful data displayed is the result of complex calculations and logic which make adding a .csv or .xlsx export difficult.

As I was looking at the page today, I realized the easiest way to create a .csv might be to extract the data out of the DOM. This had the added advantage of requiring the least amount of development.

The key HTML5 features that can be leveraged for this are Data URIs and the download attribute of the anchor tag.

Data URIs allow you to embed content into a tags. In this example, the data URI is: "data:text/csv;base64,(data here)". This indicates that the data is text, containing CSV and encoded with base64.

The download attribute tells the browser to download the content associated with the link rather than navigating to it.

This code is written in plain JavaScript, no jQuery or other library. It queries the HTML and extracts the values in the table. Multi-column cells are padded to ensure the .csv content aligns to the table. Anything that isn't numeric is enclosed in double-quotes with any embedded double-quotes escaped.

I used two loops - the first to get the data from the DOM and the second to create the .csv. It could probably be done in a single loop as well.

The HTML for the link tag is:

<a id="export-link" href="" download="nifty.csv">Export to CSV</a>

  
 /* Create a .csv of the table */
                
// Get all the rows
var trs = document.querySelectorAll("tr");
 // Declare variables
var i,l,j,k,tds,cs,m,n;
var rows = [];
var cells;
                
// Loop through all the rows
l = trs.length;
for (i = 0; i < l; i++) {
  // Get all the cells on the row
  tds = trs[i].querySelectorAll("th,td");
  k = tds.length;
    if (k > 0) {
      cells = [];
        // Loop through all the cells (including headers)
        for (j = 0; j < k; j++) {
          cells.push(tds[j].textContent);
          // Check if this cell has a colspan
          cs = tds[j].colSpan;
          m = cs - 1;
          // Pad the row to ensure everything lines up
          for (n = 0; n < m; n++) {
            cells.push("");
          }
        }
        // Add the cells as a row
      rows.push(cells);
    }
}
// At this point, rows is a two dimensional array with everything from the table

// Create a data-uri of the CSV content
var base64encode, csv = "", values;

// Loop through all the rows
l = rows.length;
for (i = 0; i < l; i++) {
   // Loop through all the cells in the row
   k = rows[i].length;
   values = [];
   for(j = 0; j < k; j++) {
     value = rows[i][j];
     // If the value is not a number, enclose it in quotes
     if (isNaN(parseFloat(value,10))) {
       value = '"' + value.replace(/"/,'\\\"') + '"';
     }
     values.push(value);               }
     // Join the cells into a CSV row
     csv += values.join(",") + "\n";
   }
}
// Create the data-uri
base64encode = "data:text/csv;base64," + btoa(csv);
// Update the link
document.getElementById("export-link").href = base64encode;

This post courtesy of Game Creek Video

I recently upgraded a Symfony 3.3 application to Symfony 4

Part of the upgrade was loading the DataFixtures.

Symfony 4 recommends you put all your DataFixtures in a single file. I'll get around to that later. However, due to the way I organized the file system for the project, the Doctrine Fixtures Loader could not find the demo data.

To resolve the issue, I created a services_dev.yaml file with the following:

services:
    App\DataFixtures\Demo\:
        resource: '../src/DataFixtures/Demo/*'
        tags: [ doctrine.fixture.orm ]

Once I added this file to the development server, the data loaded fine.

Ref: https://symfony.com/doc/master/bundles/DoctrineFixturesBundle/index.html#multiple-files

If your iPhone is not connecting to your NetGear WNDR4000 WiFi router and the message is "Incorrect password", check the access control settings on the router.

I made a change to the router configuration and managed to enable access control which allowed only one device to log in.

Oops

My latest adventure has been to upgrade a web application from Symfony 3.3 to 4. All the pages load and I am starting to test execution.

This error came up and I scoured the code for instances of AppBundle

Then I checked the database.

One of the attributes is custom_attributes, which is a JSON column. Sample content:

[{"#type":"AppBundle\\Entity\\CustomAttribute","key":"expiration","value":"2018-02-26","valueValidExpiration":true,"valueValidChannels":true},{"#type":"AppBundle\\Entity\
\CustomAttribute","key":"channels","value":6,"valueValidExpiration":true,"valueValidChannels":true}]

I am using https://github.com/dunglas/doctrine-json-odm to provide JSON data within entities.

To change AppBundle to App, I used:

dev=# UPDATE asset SET custom_attributes= REPLACE(custom_attributes::TEXT,'AppBundle','App')::json;
UPDATE 25
dev=# UPDATE model SET custom_attributes= REPLACE(custom_attributes::TEXT,'AppBundle','App')::json;
UPDATE 9