Blog about Symfony and more…

This week we are optimizing Makoondi to make it load faster. I am using the page speed plugin by Google. One of the first advices it gives is Enable gzip compression, Compressing the following resources with gzip could reduce their transfer size by about two thirds.

I got it working pretty eadily for the .html files, just by adding this to http.conf:

AddOutputFilterByType DEFLATE text/css text/html text/javascript application/x-javascript application/javascript

But it was not working for .js and .css files, no wonder how many filetypes I could add to the list. It took me a while to find the solution here by Sören Kuklau, and I am so happy that I will replicate it here :) I added this chunk of code to my http.conf:

<Directory /var/www/makoondi/web/js/>
    AddHandler application/x-httpd-php .js php_value auto_prepend_file gzip-js.php php_flag zlib.output_compression On
</Directory>
<Directory /var/www/makoondi/web/css/>
    AddHandler application/x-httpd-php .css php_value auto_prepend_file gzip-css.php php_flag <lib.output_compression On
</Directory>

Then in the …/web/css/ directory I put this simple php file called gzip-css.php:

<?php
       header("Content-type: text/css; charset: UTF-8");
?>

And in the …/web/css/ directory gzip-js.php:

<?php
        header("Content-type: text/javascript; charset: UTF-8");
?>

What is that CSRF thing?

CSRF, or Cross-site request forgery, is a form of attack that takes advantage on the trust that a web application has on the user’s browser. If a different site than yours makes a user do a request to your site, since the user has a cookie session for your site, if you don’t have further protection, your application will trust the user’s browser and execute the request. But the user is not doing it voluntarily.

Wikipedia explains it better, but basically if you are logged in into your electronic banking account and you visit another web, you won’t like that that page makes your browser perform a request to your bank to transfer your money to somebody else’’s account. And you don’t want to be the programmer of that electronic banking app ;).

How to prevent it? One way is CSRF tokens. This is the solution that symfony employs.

What is a CSRF token?

A CSRF token is a key generated by the server that depends on a secret key, a session id and the class of the form. You provide it with a form as a hidden value, and you expect the browser to send that key back to you when the form is submitted. As the attacker doesn’t know your secret key, he cannot generate a valid token. Simple, uh?

Let’s take a look at the relevant pieces of code of sfForm:

$CSRFProtection    = false,
$CSRFSecret        = null,
$CSRFFieldName     = '_csrf_token',

The first variable determines whether the CSRF protection is activated. CSRFSecret holds the secret key of theapplication, and CSRFFieldName is… the name of the field in the form.

/**
* Returns a CSRF token, given a secret.
*
* If you want to change the algorithm used to compute the token, you
* can override this method.
*
* @param string $secret The secret string to use (null to use the current secret)
*
* @return string A token string
*/
public function getCSRFToken($secret = null)
{
  if (is_null($secret))
  {
    $secret = self::$CSRFSecret;
  }

  return md5($secret.session_id().get_class($this));
}

See? it is doing an MD5 encryption with your session_id, your secret and the name of the class of the form. The session is important, because if the form is left open enough time to let the session expire, the validation will fail.
/**
* Adds CSRF protection to the current form.
*
* @param string $secret The secret to use to compute the CSRF token
*/
public function addCSRFProtection($secret)
{
  if (false === $secret || (is_null($secret) && !self::$CSRFProtection))
  {
    return;
  }

  if (is_null($secret))
  {
    if (is_null(self::$CSRFSecret))
    {
      self::$CSRFSecret = md5(__FILE__.php_uname());
    }

 $secret = self::$CSRFSecret;
 }

 $token = $this->getCSRFToken($secret);

 $this->validatorSchema[self::$CSRFFieldName] = new sfValidatorCSRFToken(array('token' => $token));
 $this->widgetSchema[self::$CSRFFieldName] = new sfWidgetFormInputHidden();
 $this->setDefault(self::$CSRFFieldName, $token);
}

This function sets a secret key if there is not $secret parameter depending on the uname of the machine, but the interesting thing here is that the validator for the field is set to sfValidatorCSRFToken:

class sfValidatorCSRFToken extends sfValidatorBase
{
  /**
  * @see sfValidatorBase
  */
  protected function configure($options = array(), $messages = array())
  {
    $this->addRequiredOption('token');

    $this->setOption('required', true);

    $this->addMessage('csrf_attack', 'CSRF attack detected.');
  }

  /**
  * @see sfValidatorBase
  */
  protected function doClean($value)
  {
    if ($value != $this->getOption('token'))
    {
      throw new sfValidatorError($this, 'csrf_attack');
    }

  return $value;
  }
}

Embedded forms in symfony are a very useful tool, but ¿how to add them dynamically from the view using AJAX?

Simple form

Let’s take a simple applicaton. There will be flashcards with a question and its answer. The form for creating these cards could be this:

With an embedded form

We may want to offer an image as a hint, embedding a PictureForm:

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);
    $this->embedForm('picture', new PictureForm());
  }
}

Dynamic embedded forms

¿What if we are interested in having more than one image as hint, a variable number of them?

In this case we need to write more code. Let’s start with the form (CardForm.class.php en my case):

class CardForm extends BaseCardForm
{
  public function configure()
  {
    unset($this['cardset_id'],$this['usercard_list']);

    //Embedding at least a form
    $pictures = $this->getObject()->getPictures();
    if (!$pictures){
      $picture = new Picture();
      $picture->setCard($this->getObject());
      $pictures = array($picture);
    }

    //An empty form will act as a container for all the pictures
    $pictures_forms = new SfForm();
    $count = 0;
    foreach ($pictures as $picture) {
      $pic_form = new PictureForm($picture);
      //Embedding each form in the container
      $pictures_forms->embedForm($count, $pic_form);
      $count ++;
    }
    //Embedding the container in the main form
    $this->embedForm('pictures', $pictures_forms);
  }
}

This code will be valid for new objects (we will embed only a form to start with) and for objects with already embedded pictures.

Now we will write an AddPicture() function that will be very useful:

public function addPicture($num){
  $pic = new Picture();
  $pic->setCard($this->getObject());
  $pic_form = new PictureForm($pic);

  //Embedding the new picture in the container
  $this->embeddedForms['pictures']->embedForm($num, $pic_form);
  //Re-embedding the container
  $this->embedForm('pictures', $this->embeddedForms['pictures']);
}

This function adds a PictureForm to the main form. This is useful for two reasons (we will see this in more detail later):

1. When adding a picture from an AJAX call, we will call an action that will run this function to obtain a form with the embedded form
. Them, that action will render only what we need. And that will be what we will inject in the view.
2. When we (bind) this form, we need the form that is going to be bounded to the user-written data to have the right number of embedded pictures. That is, if we are going to bind a user-introduced form with a given number of embedded pictures, the form object must have the same number of embedded forms. For that reason, we will overload the bind function to add as many embedded pictures as we need.

Step by step. First: in the view, we will add this AJAX function. I use Jquery, it is not difficult to write the same thing in Prototype:

<script type="text/javascript">
var pics = <?php print_r($form['pictures']->count())?>;

function addPic(num) {
  var r = $.ajax({
    type: 'GET',
    url: '<?php echo url_for('card/addPicForm')?>'+'<?php echo   ($form->getObject()->isNew()?'':'?id='.$form->getObject()->getId()).($form->getObject()->isNew()?'?num=':'&num=')?>'+num,
    async: false
  }).responseText;
  return r;
}
$().ready(function() {
  $('button#add_picture').click(function() {
    $("#extrapictures").append(addPic(pics));
    pics = pics + 1;
  });
});
</script>

And after the form a button to add forms and a place to inject them after the AJAX call:

<?php echo $form ?>
<div id="extrapictures"/>

<tr><td><div><button id="add_picture" type="button"><?php echo "Añadir otra imagen"?></button></div></td></tr>

In the action, we write the this executeAddPicForm() function:

public function executeAddPicForm($request)
{
  $this->forward404unless($request->isXmlHttpRequest());
  $number = intval($request->getParameter("num"));

  if($card = CardPeer::retrieveByPk($request->getParameter('id'))){
    $form = new CardForm($card);
  }else{
    $form = new CardForm(null);
  }

  $form->addPicture($number);

  return $this->renderPartial('addPic',array('form' => $form, 'num' => $number));
}

Basically, this function creates a form and calls to addPicture() to have a new form with an embedded form withh the right number. Then we run a partial addPic with this form and the number of embedded form to show. This partial has the following code:

<?php echo $form['pictures'][$num]['id']->render()?>
<?php echo $form['pictures'][$num]['file']->renderRow();?>

With this we have a button that adds new subforms. But, ¿what about saving? As we said before, we will need bid to do part of the task. Its new code will be:

public function bind(array $taintedValues = null, array $taintedFiles = null)
{
  foreach($taintedValues['pictures'] as $key=>$newPic)
  {
    if (!isset($this['pictures'][$key]))
    {
      $this->addPicture($key);
    }
  }
  parent::bind($taintedValues, $taintedFiles);
}

And with this change we can use this form as a normal one.

PS: I must give people credit about this one. I looked at several places for this info and this post was very useful. It is basically the same, but more focused in adding fields synamically instead of embedded forms.

PS2: See also this post and this one about this topic.

When  $form->bind() throws this error:

This form is multipart, which means you need to supply a files array as the bind() method second argument.

It means that at least one field in the form is multipart. That means that the form contains files. That’s why bind needs a second argument, usually with the form $request->getFiles($form->getName());

$this->form->bind($request->getParameter($this->form->getName()), $request->getFiles($this->form->getName()));

In a previous post I explained bind() more in depth

La documentación de formularios de Symfony va mejorando, pero sigue habiendo muchos puntos oscuros. Los formularios siguen siendo una de partes más complejas de entender de Symfony. Hay una forma simple para hacer cosas simples, pero a la que te quieres mover de ahí, es necesario entender cómo funciona todo por dentro.

Y el punto más importante del proceso está en la función bind(). Por ejemplo, si hemos hecho un post validator y no funciona, si estamos trabajando con formularios empotrados (embedded), o simplemente si algo no funciona, necesitaremos entender qué está pasando en este método. Así pues, ¿Qué hace bind()? Según la documentación de formularios de Symfony 1.2 bind() hace lo siguiente:

Cuando el formulario se asocia (bind) a los datos externos usando bind(), el formulario pasa al estado «bound» (ya veremos que esto no significa gran cosa) y se ejecutan las acciones siguientes:

* Se ejecuta el proceso de validación
* Los mensajes de error se almacenan en el formulario para que estén disponibles para la template
* Los valores por defecto del formulario son reemplazados por los datos que ha introducido el usuario.

Vamos a verlo con más detalle. Una función de procesado de formulario típica (bind(), y si es válido, save()) tiene esta forma:

protected function processForm(sfWebRequest $request, sfForm $form)
{
  $form->bind($request->getParameter($form->getName()), $request->getFiles($form->getName()));
  if ($form->isValid())
  {
    $card = $form->save();
    $this->redirect('card/edit?id='.$card->getId());
  }
}

Esta operación tiene dos actores: la $request y el $form. Dentro de todo lo que hay en una $request, si todo ha ido bien habrá un array. En el caso de este objeto (card), nuestra $request contiene este campo:

[postParameters:protected] => Array
        (
            [card] => Array
                (
                    [question] => elefante
                    [answer] => mamífero
                    [id] =>
                    [_csrf_token] => a288d0365a6bb4aab5a11789534a4ddb
                )
        )

Así que tenemos los datos del formulario en un array. Este array es el que enviamos a bind en el primer parámetro al hacer

$form->bind($request->getParameter($form->getName()), $request->getFiles($form->getName()));

¿Qué hace bind con esto? Miremos el código de sfForm->bind() :

 /*
 * Binds the form with input values.
 *
 * It triggers the validator schema validation.
 *
 * @param array $taintedValues An array of input values
 * @param array $taintedFiles  An array of uploaded files (in the $_FILES or $_GET format)
 */
public function bind(array $taintedValues = null, array $taintedFiles = null)
{
  $this->taintedValues = $taintedValues;
  $this->taintedFiles  = $taintedFiles;
  $this->isBound = true;
  $this->resetFormFields();

  if (is_null($this->taintedValues))
  {
    $this->taintedValues = array();
  }

  if (is_null($this->taintedFiles))
  {
    if ($this->isMultipart())
    {
      throw new InvalidArgumentException('This form is multipart, which means you need to supply a files array as the bind() method second argument.');
    }

    $this->taintedFiles = array();
  }

 try
  {
    $this->values = $this->validatorSchema->clean(self::deepArrayUnion($this->taintedValues, self::convertFileInformation($this->taintedFiles)));
    $this->errorSchema = new sfValidatorErrorSchema($this->validatorSchema);

    // remove CSRF token
    unset($this->values[self::$CSRFFieldName]);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $this->values = array();
    $this->errorSchema = $e;
  }
}

En las dos primeras líneas guarda nuestro array de valores que viene de la request en variables del objeto. La siguiente marca el formulario como bound. Es decir, que aunque haya un desastre en un punto posterior del código de bind(), si llamamos a $form->isBound() nos dirá que efectivamente está «bound». Es decir, que isBound() significa «se ha llamado a bind para este formulario» y no «el bind ha funcionado». resetFormFields() simplemente vacía el array de $this->formFields y también $form->formFieldSchema . El fieldschema es el objeto que realiza el render de un field del formulario cuando lo queremos representar.

Lo siguiente que hace es comprobar que taintedValues y taintedFiles existan y sean coherentes con el formulario (si es un formulario multipart, necesitará taintedFiles).

Tras los preparativos ejecuta una línea que es la que hace los tres puntos de la descripción de bind(). Es decir, la que hace lo que esperamos de bind():

$this->values = $this->validatorSchema->clean(self::deepArrayUnion($this->taintedValues, self::convertFileInformation($this->taintedFiles)));

De dentro a afuera:
* convertFileInformation() «convierte un array de fichero a un formato que cumple la convención de nombres de $_GET y $POST».
* deepArrayUnion() es una función recursiva que fusiona dos arrays.
* Y este array es el que se le pasa a $this->validatorSchema->clean().

Es decir, que le pasa el muerto al validatorSchema, que es el que carga con el peso del bind. Una conclusión que podemos sacar es que, si hay formularios embedded, sus métodos bind no se van a llamar, pero sí sus validadores. Así que la lógica de sus binds tiene que ir en alguna otra parte.

¿Qué hace $this->validatorSchema->clean()? llama a doClean que hace Esto:

protected function doClean($values)
{
  if (is_null($values))
  {
    $values = array();
  }

  if (!is_array($values))
  {
    throw new InvalidArgumentException('You must pass an array parameter to the clean() method');
  }

  $clean  = array();
  $unused = array_keys($this->fields);
  $errorSchema = new sfValidatorErrorSchema($this);

  // check that post_max_size has not been reached
  if (isset($_SERVER['CONTENT_LENGTH']) && (int) $_SERVER['CONTENT_LENGTH'] > $this->getBytes(ini_get('post_max_size')))
  {
    $errorSchema->addError(new sfValidatorError($this, 'post_max_size'));

    throw $errorSchema;
  }

  // pre validator
  try
  {
    $this->preClean($values);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $errorSchema->addErrors($e);
  }
  catch (sfValidatorError $e)
  {
    $errorSchema->addError($e);
  }
  // validate given values
  foreach ($values as $name => $value)
  {
    // field exists in our schema?
    if (!array_key_exists($name, $this->fields))
    {
      if (!$this->options['allow_extra_fields'])
      {
        $errorSchema->addError(new sfValidatorError($this, 'extra_fields', array('field' => $name)));
      }
      else if (!$this->options['filter_extra_fields'])
      {
        $clean[$name] = $value;
      }

      continue;
    }

    unset($unused[array_search($name, $unused, true)]);

    // validate value
    try
    {
      $clean[$name] = $this->fields[$name]->clean($value);
    }
    catch (sfValidatorError $e)
    {
      $clean[$name] = null;

      $errorSchema->addError($e, (string) $name);
    }
  }
  // are non given values required?
  foreach ($unused as $name)
  {
    // validate value
    try
    {
      $clean[$name] = $this->fields[$name]->clean(null);
    }
    catch (sfValidatorError $e)
    {
      $clean[$name] = null;

      $errorSchema->addError($e, (string) $name);
    }
  }

  // post validator
  try
  {
    $clean = $this->postClean($clean);
  }
  catch (sfValidatorErrorSchema $e)
  {
    $errorSchema->addErrors($e);
  }
  catch (sfValidatorError $e)
  {
    $errorSchema->addError($e);
  }

  if (count($errorSchema))
  {
    throw $errorSchema;
  }
  return $clean;
}

Y aquí está el quid de la cuestión. Tras asegurarse de que los datos de entrada tienen sentido, llama al preValidator, si es que hay uno definido, en preClean(). Entonces llama a la validación campo a campo. Y acaba llamando al postValidator, en postClean(). Hay que fijarse en cosas como que nuestro postValidator realmente devuelva valores, porque si no devuelve nada, si nos fijamos en el código veremos que no retornaremos los valores «cleaned» y estaramos liándola, porque entonces $this->values del formulario quedará vacío.

Resulta complicado encontrar errores así. Por eso es interesante entender qué está pasando al hacer un bind. En este post he tratado de resumirlo y de reunir el código que hay disperso por la API, para que uno pueda tenerlo a mano y entender qué puede estar pasando cuando las cosas fallan.