1. Web Servers
2. Databases Servers
3. Caches Servers

Lots of other things are also required. These all things helps in reducing the page render time when we have lesser amount of data. What if you have a terabytes of data and you need to serve it realtime Ooh that’s really difficult work to do. I will share my experience with you guy how I am doing this?

Well my thoughts are that Web servers are the processors, cache is RAM and databases are the storage. We better know that storage I/O is very slow we cannot serve too much traffic from the normal databases like mySQL, Oracle, MSSQL etc. So we need to have some sort of things those can reduce our dependencies from the storage databases.

So what are the things that we can use to increase the performance of our web applications at very low cost? So I will address the issues and then there solution for you.

1. Full text search: On a huge amount of data in relational database full text search is almost impossible so we can use the Sphinx search system. Sphinx performance of serving full text search is outstanding.
2. SQL JOIN: Store the data in denormilized form on sql queries runs very fast with less joins. It’s easy to store that data in noSQL forms like Cassandra, Hadoop ,etc. These noSQL algorithms helps in sorting of trillions of rows in few seconds.
3. Caches: We need to use the cache systems very effective like complete key management system of Memcache, MCACHE, Eaccelerator, etc. These caches can help you in most access pages and static contents to serve them from the memory faster.
4. Web Servers: Optimization of web servers are very important like we mostly use Apache web server which is more famous and its performance is outstanding on prefork and MPM Workers both thread controlling mechanism so we need to configure these process system very smartly.

The Open source OWASP( Open Web Application Security Project) is a non revenue dependent worldwide charity organization, concentrating on the improvement of web application security. Their particular mission is to highlight the application security mechanism in every way possible, so that people and organizations can be informed regarding original application security risks and their solutions. Worldwide, OWASP is free for everyone to join and take advantage or provide and help on OWASP’s security projects.

All of the information about security risks and their solutions are available under one free web portal. All of the information can be found on OWASP’s official website and all the latest information about it is available on OWASP’s blog. It has been requested to feel very comfortable to help make any changes and improvement to any of their security projects. In fact, there are hundreds of people around the world who review the changes to the site to help ensure quality and performance. They do have a ‘ Getting Started ‘ page for new users, and it becomes very easy for a new person to get involved with the information they are providing. It has been highly appreciated by OWASP team that every visitor become a member of their security portal.

Some of the projects done by OWASP are follows:

1. OWASP AntiSamy Java Project:  An API for validating rich HTML/CSS input from users without exposure to cross-site scripting and phishing attacks.
2. OWASP AntiSamy. NET Project:  An API for validating rich HTML/CSS input from users without exposure to cross-site scripting and phishing attacks.
3. OWASP Enterprise Security API (ESAPI) Project:  A free and open collection of all the security techniques that a developer needs to develop a secure web application.
4. OWASP ModSecurity Core Rule Set Project:  A project to document and develop the ModSecurity Core Rule Set.

OWASP AntiSamy Java Project

The OWASP AntiSamy project consists a handful of things. Technically, it is an API for ensuring HTML/CSS provided by user is in compliance within an application’s rules. Yet another way of stating that could be: It’s an API that assists you to make sure that your clients don’t supply any harmful cargo codes in the HTML they supply for their profile, comments, images etc… that get persisted on the server. “Malicious Code” is a temr regarding to web applications usually mean “JavaScript”. The Cascading Stylesheets (css) are only considered malicious when they are linked with JavaScript engines. Moreover, there are many situations where a completly normal HTML and CSS cab be used in many different harmful manners, AntiSamy project take care of that too.

Philosophically, AntiSamy is a departure from modern day security mechanisms to the future security mechanisms. Generally speaking, the security mechanism and user have a very useful communication which is virtually one way, for very good reasons. Leading the potential attacker know the details about the validation is considered unwise as it allows the attacker to discover and recon the mechanism for weaknesses. These kinds of information leaks can also become harmful in many ways that a user cannot expect. A login mechanism that tells the user, “Username invalid” leaks the fact that a user by that name does not exist. There are more things like the Username example which we does not consider to any worth. Attacker can use a dictionary and a phonebook to remotely check out the list of valid usernames. Using this information, an attacker could use  a brute-force attack (very popular attack these days) or massive account lock denial-of-service. We get that. Unfortunately, that’s just not very functional in this situation. Typical Internet users are largely pretty bad when it comes to writing HTML/CSS, so where do they get their HTML from? Usually they copy it from somewhere out on the web. Simply rejecting their input without any clue as to why is jolting and annoying. Annoyed users go somewhere else to do their social networking.

The Owasp Enterprise Security Api

ESAPI (The Owasp Enterprise Security Api) is a web security control application library which provide ease to the developers to program applications with low risks. The ESAPI liabriries are basically designed for programmers to use it to retrofit security into their existing applications easily. ESAPI libraries are also very helpful for new development’s security foundations. Enabling for language-specific differences, all OWASP ESAPI versions have the same basic design:

There’ve been a set of interfaces for security control, they define for example, types of parameters that are passed to types of security controls.Each security control system is implemented with a reference guide for implementation. The logic is not both organization and application specific, string-based input validation is a good example. There could be different and custom implementations for each of your security control. In these classes, it may contain application logic which may be designed by or for your organization according to it’s requirement. Enterprise authentication is a good example.
This project source code is licensed under the BSD license, which is very permissive and about as close to public domain as is possible. The project documentation is licensed under the Creative Commons license. You can use or modify ESAPI however you want, even include it in commercial products

OWASP ModSecurity Core Rule Set Project

ModSecurity™
A firewall web application engine which does not offer a lot of protection on its own. ModSecurity™ is to be configured under rules in order to become beneficial. SpiderLabs from Trustwave is serving a free certified rule set to provide maximized advantage for users of ModSecurity™ 2. x. It is not like a intrusion detection application or its preventation system, that rely on specific signature to capture vulnerabilities.
The Core Rules are providing many similar type of protections against known and unknown vulnerabilities which are found in several web applications (which are mostly custom coded so they become unknown). The Core Rules are to be used for step-by-step deployment for ModSecurity™.

The Core Rules Content

To provide numerous similar sort of web applications’ protection, Core Rules use the following methods:

1.     HTTP Protection – detects violation of the HTTP protocol as well as locally defined usage policy.
2.     Real-time Blacklist Loockups – utilize 3rd Party IP Reputation.
3.     Web-based Malware Detection – identifies malicious web content by examine towards the Google Safe Browsing API.
4.     HTTP Refusal of Service Protections – defense against HTTP Flooding and Slow HTTP DoS Attacks.
5.     Common Web Attacks Protection – detecting common web application security attack.
6.     Automation Detection – Detecting bots, crawlers, scanners and other surface harmful activity.
7.     Integration with AV Scanning for File Uploads – detects harmful files uploaded through the web application.
8.     Tracking Sensitive Data – Monitors Credit Card usage and blocks leakages.
9.     Trojan Protection – Detecting access to Trojans horses.
10.  Identification of Application Defects – alerts on application misconfigurations.
11.  Error Detection and Hiding – Disguising error messages sent by the server.

1. Server Generation Time
2. Network latency ( Bandwidth )
3. Browser Rendering Time

These three major areas in performance effect the time. We need to have focus on all there 3 variables to improve the performance of the website. There are many tools help you in gagging the performance like webpagetest.org and Google mod_pagespeed an Apache module.

Google have weight for performance in his Algorithm. Its says if the website loading time increasing so its mean that user will be not happy with our services. On daily basis when we are surfing websites it is clear in our minds that when the site is not opening in time then mostly we don’t want to browse another page. The most website is fast the most it help user to stay on it.

When the Google start working on website performance we have notices that 25% to 60% websites improved their performance. If you don’t have the good performance of your website and your ranking is not increasing Google its mean that you have a performance issue. Improve your site performance and see the results.

Twitter build the flockdb which is open source used in the distributed environment, it is also called the fault-tolerant graph database. It is used to manage the data huge amount of data on web environment. Twitter is using this database to manage the relationships of users like how is following whom and stores the data 1st relation second relation and so one.witter says that it is a very simpler from the other graph databases. Its designed is horizontally scaleable for high traffic websites. This database is licensed under Apache. Twitter new gizzard tool is focused on the eliminating the pain of high end databases and queries.

This database will going to join the noSQL family soon like Cassandra and Hadoop. I think twitter efforts will help new business engaging in the social graph it will be easy and cost effected.

Technical details

FlockDB distributed environment graph database. Its benchmark in 2010 cluster of FlockDB stores more then 13 billion data and on maximum traffic of 20K+ w/sec and 100k+ r/sec. Which is the ultimate thing I ever seen? It is designed by using Java, sbt, thrift apache, Thrift is known as the extreme performer on the web.  Its supports

• 80K/sec add/update/remove operations it can do
• Complex set arithmetic queries can be designed and executed
• paging through query result sets containing millions of entries
• You can archive and  restore archived data anytime
• Replication with horizontal scaling
• Data migration without shutting down the database

For elements like <ol> <li> <ul>, child selectors should be used instead of descendant selectors. Child selectors support is available since IE7. For modifying any <li> item in <ol> following syntax can be used, ul > li {/*new style*/}

Hope this will help you design good site with optimized css architecture

Defining an inline style for element is less optimized, unless the style is only used once. If the style is used multiple times it is it is better to put style in external css file.

For example if your code looks like this

<p style="font-size:1em;color:white;">Lorem ipsum dolor sit amet, consectetur adipiscing elit</p>
<p style="font-size:1em;color:white;">consectetur adipiscing elit</p>
<p style="font-size:1em;color:white;">Lorem ipsum dolor sit amet</p>

than you should move style into a block like this

<style type="text/css">
#para p{font-size:1em;color:white;}
</style>

Now the HTML will look a bit cleaner

<div id="para">
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit</p>
<p>consectetur adipiscing elit</p>
<p>Lorem ipsum dolor sit amet</p>
</div>

So if you start optimizing your site speed, specially css, consider this option. For this you will need to move all <style> blocks and “style” tag to a css file. This will reduce the page size smaller and quick to download. This will also reduce code to content ratio which is beneficial for SEO.

First we will see how GA script impacts the performance.

1. External Script: Means it will add extra HTTP call and DNS resolution.
2. Its JavaScript: Placement is crucial as javascript can be a blocking call.

How to make it faster:

1. External Script: We can’t do much about it. Google usually have very good DNS resolution time and content download time, so there is not much to worry about it. Also as most of the sites have GA script, so mostly the js will be served from local cache.
2. Its a JavaScript: Older version of GA are blocking calls, so if you are using it than be sure to place the GA code just before the </body> tag. Blocking call will halt any download, rendering etc unless js file is downloaded and executed.
3. Upgrade to Newer Version: If you need to place GA script in <head> or higher in the <body> than you should upgrade the GA version to latest version which allows async download. Async call downloads the JavaScript in a manner that it does not block any other resources from download. In my personal opinion even with async call you should place the GA to just above the</body>. However it is also safe to place GA in <head>.

If you want to upgrade the version to async version than you can start with http://code.google.com/apis/analytics/docs/tracking/asyncMigrationExamples.html

I recently experienced such an incident where an image was added. The image was visible at every page of the website. The issues with the image were

• External Site (Extra DNS to be resolved)
• No expiry headers defined
• Image not optimized

This caused our page load time to increase by almost 0.5 seconds. Notice the red circle (a sort of) where the image was added and site performance graph went up.

After a few days image was fixed by adding proper expiry headers and optimizing file size. The +ve impact was immediate as see in site performance graph.

If the access log file size reaches to around 4GB on windows machine, apache will be very slow to respond and will hurt the web application performance badly. You should keep checking log file size regularly to make sure its within acceptable range. Access log file is normally found in logs/access.log under the apache installation.

Apache provides built in functionality to rotate these logs on daily basis so it does not grow much and create performance issues to server. If we set it up, new file will be created for each day. Following is method to setup log rotation on Windows with WAMP, Apache 2.2.11.

Log rotation can be done at two levels, one server level and second on virtual host level. To do a server level log rotation you will need to modify /conf/httpd.conf file under apache installation. To do a virtual host level rotation will need to modify file \conf\extra\httpd-vhosts.conf file.

For server level log rotation open file /conf/httpd.conf and find line like
CustomLog "C:/Program Files/wamp/logs/access.log" common
The line defines the log file. Now we will define log rotation in following manner
CustomLog "|bin/rotatelogs.exe C:/Program Files/wamp/logs/access_%y-%m-%d.log 86400" common
here |bin/rotatelogs.exe tells that logs should be sent to rotatelogs.exe program which is located under apache installation in bin folder (make sure the exe file exists and have permissions).

The C:/Program Files/wamp/logs/access_%y-%m-%d.log tells the location of new log files. Here %y is year, %m is month and %d is day so the file name remains unique. 86400 is time span defined in seconds after which new file is generated, in this case its 24 hours (24x60x60) means log should rotate daily.

Be sure that you user have permission to write a file on given location (C:\ in this case). Also be sure that there is no space in file path given as it will create issues in creating new file.

After making the changes save file and restart apache. Now check if new file is created properly on access any web pages on this server. The new file should be created with name like “access_10-08-18.log” assuming that today is (Aug, 18 2010).

If you have to do same thing for a particular virtual host than you should add following line in <VirtualHost xxxxxxxx> definition.CustomLog "|bin/rotatelogs.exe C:/Program Files/wamp/logs/access_%y-%m-%d.log 86400"

This should help improving performance of the web server as it has to write to a small file, secondly it is easy to download and analyze small logs as compared to huge files.

Browsers allow parallel connections to a single hostname. This allows files to download faster in parallel and very helpful for web application performance. When JavaScript file is called browsers halts the parallel connections to all hosts. This is because JavaScript can alter the document dom with document.write().

This can badly hurt the front end performance of a web application. To avoid this all calls to JavaScript files should be moved to bottom of the page, just before end to BODY tag.

If it is not possible to move JavaScript calls to bottom, one should use DEFER attribute when calling script. Example code for this is
<script src="script.js" type="text/javascript" defer="defer"></script>
Here defer tells that JavaScript does not contain any document.write() and browser should continue rendering. Problem with defer is that FireFox does not support it, Internet Explorer supports the defer but not to as desired. Still its batter to use it.

For faster page loadeds its recommended that JavaScript should be moved at bottom of the page, so parallel downloads are not blocked by JavaScript files.

This post is part of our “Web Application Performance“. You can find more exciting post to improve performance of a web application in Web Application Performance category.