Data Science for Everyone

An introductory video lesson on Big Data, the need, necessity, evolution and contributing factors.

This is presented by Skill Sigma as part of the "Data Science for Everyone" series.

1. Know the trending languages(past & present) and their comparision with other languages: @ https://www.tiobe.com/tiobe-index/
2. Found an interesting website? identify the technologies used to build website by crome/firefox extension @ https://wappalyzer.com/download
3. Wanna develop out standing website? Need an inspiration? Study code & have fun: @ https://codepen.io/pens/
4. Stuck with bug for days? Get some help @ https://stackoverflow.com/questions/tagged/php
5. Need Infotainment? Chat with fellow programmers @ https://chat.stackoverflow.com/
6. Know software concepts @ https://softwareengineering.stackexchange.com/
7. Compare abilities & limitations of different languages @ https://codegolf.stackexchange.com/
8. Know the best practices by code sharing @ https://codereview.stackexchange.com/
9. Know different ways to achive similar objectvies by studying code @ https://github.com/showcases & https://sourceforge.net/
10. Know how much does a technology worth, how much you are earning in comparision to your peers with different experiences, different locations, different companies @ https://www.payscale.com/wizards/choose.aspx

10 Essential SQL Tips for Developers:

SQL is yet another essential language for developers wishing to create data-driven websites. However, many developers are unfamiliar with various aspects of SQL; so in this article, we'll analyze ten essential tips.

1. Use The Right Language:

Web developers often have a plethora of languages at their disposal. It is crucial for developers to use the proper language for the job.

2. Secure Your Code:

Databases store valuable information. Because of this fact, databases are often prime targets for attack. Many developers are unaware that their code has critical security vulnerabilities, which is a very scary fact not only for clients, but also for you. Currently, developers can be held legally accountable if their own personal negligence results in a database security risk that is then exploited.

 How Can You Write Secure Code?

The solution is often DBMS specific; that is, it varies between MySQL, Oracle or SQL Server. In PHP with MySQL, for example, it is usual to escape parameters using the function mysql_real_escape_string before sending the SQL query. Alternatively, you can utilize prepared statements to "prepare" your queries. Make it your mission to understand the DBMS with which you are working and the inherent security issues.

SQL injection isn't the only security vulnerability for databases and developers to worry about, however, it is one of the most common methods of attack. It is important to test your code and be familiar with the latest security issues for your DBMS in order to protect against attacks. 

3. Understand Joins:

Single table SQL select statements are rather easy to write. However, business requirements often dictate that more complex queries must be written. For example, "find all orders for each customer, and display the products for each order". Now, in this particular situation, it would be likely that there is a customer table, an order table, and an order_line table (the last would be to resolve a possible many-to-many record relationship). For those who are slightly more familiar with SQL, it is readily apparent that a table join, actually, two table joins will be required for this query. Let's look at some sample code.

4. Know Your Data Types:

In SQL, typically each table column has an associated data type. Text, Integer, VarChar, Date, and more, are typically available types for developers to choose from. 

5. Write Compliant Code:

All programming languages have standards which web developers should be aware, and SQL isn't any different. SQL was standardized by ANSI and then ISO, with new revisions to the language being occasionally submitted. The latest revision is SQL:2008, although the most important revision that developers should be aware of is SQL:1999. The 1999 revision introduced recursive queries, triggers, support for PL/SQL and T-SQL, and a few newer features. It also defined that the JOIN statements be done in the FROM clause, as opposed to the WHERE clause.

6. Normalize Your Data:

Database normalization is a technique to organize the contents of databases. Without normalization, database systems can be inaccurate, slow, and inefficient. The community of database professionals developed a series of guidelines for the normalization of databases. Each 'level' of normalization is referred to as a form, and there are 5 forms, total. First normal form is the lowest level of normalization, up to fifth normal form, which is the highest level of normalization.

First Normal Form (1NF): The most basic level of data normalization, first normal form requires the elimination of all duplicate columns in a table, and also requires the creation of separate tables for related data, and identification of each table with a primary key attribute.

Second Normal Form (2NF): Meets all the requirements of first normal form, and creates relationships between tables using foreign keys.

Third Normal Form (3NF): Meets all the requirements of second and first normal forms, and removes all columns that are not dependent upon the primary key. Third normal form also removes all derived attributes, such as age.

Fourth Normal Form (4NF): Fourth normal form adds one additional requirement, which is the removal of any multi-valued dependencies in relationships.

Fifth Normal Form (5NF): Fifth normal form is a rarer form of normalization, in which case join dependencies are implied by candidate keys (possibly primary key values).

The Benefits of Normalization:

Now, without venturing too far into database theory, let's simply focus on the benefits of normalization. As the data progresses through the normalization forms, it becomes cleaner, better organized, and faster. Now, with a small database that has only 5 tables and 100 rows of data, this won't be readily apparent. However, as the database grows, the effects of normalization will become much more apparent with regards to speed and maintaining data integrity. However, there are some situations in which normalization doesn't make sense, such as when normalizing the data will create excessively complex queries required to return the data.

7. Fully Qualify Your Database Object Names:

Now, this is a commonly ignored point; in fact, all the sample code I've demonstrated in this tutorial has essentially violated this tip. In terms of database development, a fully qualified object name looks as follows: DATABASE.schema.TABLE. Now, let's look at why fully qualified names are important, and in what situations they are necessary. The purpose of a fully qualified object name is to eliminate ambiguity. Beginning developers rarely have access to multiple databases and schemas, which complicates the issues in the future. When a given user has access to multiple databases, multiple schemas, and the tables therein, it becomes crucial to directly specify what the user is attempting to access. If you have an employee table, your boss has an employee table, and the schema that your web application is running on has an employee table, which are you really attempting to access? 

8. Understand Indexing:

A database index is a data structure that improves the speed of operations on a database table. Indexes can be created using one or more columns of a database table, providing the basis for both rapid random look ups and efficient access of ordered records. Indexing is incredibly important when working with large tables, however, occasionally smaller tables should be indexed, if they are expected to grow. Small tables that will remain small, however, should not be indexed (for example, if your book is 1 page, does it make sense to turn to the index?)

9. Properly Use Database Permissions:

When working with a database that has multiple users, it is important to properly handle various database permissions. Obviously, most databases have an administrator user, but does it always make sense to run your queries as the administrator? Additionally, would you want to provide all your junior developers and users with your administrator credentials in order to write their queries? Most likely not. The various possible permissions for your database depend on your DBMS, but there are common themes between them.

10. Know Your DBMS Limitations:

Databases are powerful tools, however, they aren't without limitations. Oracle, SQL Server, and MySQL all have unique limitations on things such as maximum database sizes, maximum number of tables, and others. Many developers unknowingly choose a DBMS solution for their project without planning or considering the later requirements of their database.

Conclusion:

In this article, we reviewed 10 essential tips for SQL developers. However, there are many other useful SQL techniques that could be mentioned; so please leave your thoughts in the comments, whether you think this article covered all the essential topics, or you think one was left out. Keep developing, and remember, the code you write supports the internet infrastructure, and without you, the internet would not be as successful as it is.

SQL tips 4:
Avoid INDEX, unless you need to retrieve information quickly. Index will slower insert and update data query.

The another way is using sub query

Select MAX(salary)
FROM employee
WHERE salary IN
(Select top 2 salary from employee where salary is not null order by salary desc)

==============================================
SQL tips 5:
NULL and ZLS (zero length string) both are different.

NULL means data is unknown or missing.

NULL should be checked with IS NULL or IS NOT NULL only. Comparison operator equal to (=) or not equal to (<>) should not be used.

ZLS should be checked with two single quotes  ('') together.                        

==============================================
SQL tips 6:
Always use primary key with integer field and unique key with string or text field.

This is not rule, it is best practice.

Truncate:

• Truncate command will delete the Table data only, it keeps the table schema as it is.
• Table data Can be rollback
• Example : TRUNCATE TABLE User;

Drop:

• Drop command will delete both Table data table schema.
• Table data Can be rollback
• Example : DROP TABLE User;

Delete:

• Delete command will delete the particular row based on the criteria in Where clause. If where clause is not mention in the squre, it will delete entire data, that is equalent to Truncate command action.
• Once delete the data, data can not be rollback. Make sure to include where clause in the delete query.
• Example : DELETE FROM User WHERE userid = 5;

Note: This concept will work only for MS SQL. It may be different or reverse in oracle sql and MySQL

SQL tip 1:

• Instead of Full Outer Join, you can use Union All.
• Difference is, Full Outer Join provides results side by side.
• Union All provides result s one by one.
• Note: MYSQL does not support Full Join.                

SQL tip 2:

• To get column name of a table use following sql query
• Select column_name from information_schema.columns
• Where table_name = 'student'
• Here 'student' is name of your table. It must be single quoted
• If you want to complete information about table then use following queries.
• Select * from information_schema.columns
• Where table_name = 'student'

SQL tip 3:

• Avoid Order By clause unless you need result in ascending or descending order.
• It will be slower your query execution.                        

Database Normalization is the process of reducing duplication in database by decomposing the table

1. Normal Form (1NF):

• Every attribute is atomic, Each cell to be single valued
• No multivalued attribute: Example subject  ( C, C++ )
• No composite attribute: Example Address (392, XYZ street, Bangalore, 560001).  
• Entries in a column are same type
• Rows must be uniquely identified using UNIQUE ID or Add more column to make unique

2. Normal Form (2NF):
Table must not contain redundancy(Unnecessary repeating information).

3. Normal Form (3NF):
All fields can be determined only by Key in the table and not other column.

3.5. Normal Form - Boyce Codd Normal Form (BCNF):
Even when a database is in 3rd Normal Form, still there would be anomalies resulted if it has more than one Candidate Key. Table should not have multiple candidate key. Database should be only one primary key

4. 4NF (Fourth Normal Form):
Table instance contains two or more, independent and multivalued dependencies on Primary key

5. 5NF (Fifth Normal Form):
Table cannot be decomposed into any number of smaller tables without loss of data