Symfony Scalability Enhancements

Registered by Darlene McCullough

To support its continued evolution and growth, Sahana Agasti has recently started the development of a new series (2.0) on a new framework: Symfony/Doctrine. While the move to an ORM was a must for the future of the application, it has introduced a number of performance issues including poor query optimization with forms. This problem definition hopes to address this issue and improve the overall performance of the application to enable its use in large-scale deployment scenarios.

Blueprint information

Status:
Complete
Approver:
Chad Heuschober
Priority:
Low
Drafter:
Chad Heuschober
Direction:
Approved
Assignee:
CUNY SPS
Definition:
Approved
Series goal:
Accepted for mayon
Implementation:
Implemented
Milestone target:
milestone icon 2.1.0
Started by
Darlene McCullough
Completed by
Chad Heuschober

Related branches

Sprints

Whiteboard

Summary
=======
To support its continued evolution and growth, Sahana Agasti has recently started the development of a new series (2.0) on a new framework: Symfony/Doctrine. While the move to an ORM was a must for the future of the application, it has introduced a number of performance issues including poor query optimization with forms. This problem definition hopes to address this issue and improve the overall performance of the application to enable its use in large-scale deployment scenarios.

Example
-----------
The NYC OEM (see below) uses Sahana Agasti for its sheltering plans. These plans would require the data management of nearly 750,000 persons with associated metadata for a single event. Over time, these numbers would be expected to grow. Currently, some of the forms generated by Doctrine models produce upwards of 60 individual queries to the database when a single, well-executed join query might return the same results. The extraneous queries slow down response times and prevent the system from being used at the projected capacity levels.

Description and Constraints
======================
Right now, the only constraint is DQL and the limited subset of database functionality that it provides. The only time we use DoctrineRawSQL is when we need to use a subquery. It is our hope that those with symfony and/or sql experience can work through the models to locate opportunities for a reduction in the total number of queries executed and/or generic performance optimizations.

Extra Credit
==========
    * Finding a way to not have to do this as a manual process and have our models and forms work more intelligently
    * Executing subqueries using Doctrine DQL but can't find a subquery syntax that can provide the equivalent of:

SELECT t1.id, t1.timestamp, t1.status FROM table AS t1 INNER JOIN (SELECT t2.id, MAX(t2.timestamp) AS mxts FROM table GROUP BY t2.id) AS t3 ON t1.id = t3.id AND t1.timestamp = t3.mxts ;

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.