SQL is really boring do we need it?

[Wallstyouth]

How do you guys feel aboue SQL in general is it something we in this industry really need to know? I find SQL so boring and mundane other than basic joins I dont really have the drive to learn much more about it. Am I shooting myself in the foot for not taking SQL more seriously?

 

[Andy Nguyen]

Obviously, not everyone has to do SQL at work. And for those who do, the level of SQL knowledge required is different from place to place.

So if JOIN is the most complex SQL you do at work, then I understand why you don't see a need to learn more.

I see it as an edge you try to gain over others. The more you know, the better.

I'll share with you my story so you can take it as an incentive to learn more.

When I went for interviews during the summer, of the 4 firms I interviewed with (CS,BS,DB,UBS), CS and DB were specially asking about my familiarity with large database, SQL, etc

In the case of CS, they were expanding the product space they cover and the amount of data grows 10 folds. So they were overwhelmed, understaffed and desperate for help with the data. So much that they added a guy from their IT dept just because he has been helping them with the datafeed. Part of the daily ritual is to make sure the data is checked, cleaned, corrected so that they can calibrate the portfolio correctly everyday.

Now, as in DB case, they need someone besides finance knowledge, programming skills is familiarity with SQL so he can build, manage the data that feeds into models.

Then, I thought the definition of "familiarity with SQL" means knowing how to use SELECT, JOIN, WHERE, etc queries...

But it turns out those are very basic things that most people can pick up in a very short time.

Now, my definition of "familiarity with SQL" means knowing how to write stored procedures (SP) that contains logic, tasks, complex queries to do complicated things. It means to write macro that read and write a large block of data from your database into your Excel model. Then read and write the data in XML format, run overnight jobs to update, send report, etc.
Simple queries are not cutting it anymore.

As I learn more about SQL, I move pass the SELECT, JOIN part and learn to love SQL. It is like a programming language to me now. Stored procedure is now looks like my C++ programs with IF, ELSE, CASE, WHEN, etc

Half of the time, I work with all the pricing, calibration models in VBA and when the numbers come in, I get a good idea of when a number does not look correct. When you calibrate correlation curve so many times, you have a sense of what it should look like, how the correlation smile, skew should look and when it does not look like what you expect, you know right away what part of the data causes that effect. You then can look into that specific date to see what happened in the market that cause your model to behave like that.

I learn alot about SQL the past 1,2 months and I think it's very important. Everywhere you go, you will have to deal with a huge amount of data one way or another. If you are familiar with it, you can make good use of it. That will become a very marketable skills.

Employers can hire database specialists to do this task but if you are able to do that in your job, it will increase your value.

There are lot of people here that know more about SQL than I do. When i have something I need to ask, I usually turn to Alain, Chris or Joe. (after I couldn't find what I need on google, of course

I believe the key to success on Wall Street is to learn as much as you can and be better at what you do than others.

 

[kean]

SQL seems to be simple but it can very complicated when those stuff as Andy mentioned. Another issue comes SQL when we handle large data is cleaning the data. Data integrity is a key to model building. To know it or becoming a super user is different from mastering it.

I tested S-Plus personally and I think S-plus can do a better job in terms of feeding data into a model from connecting to a database to coding. I wonder if someone can tell me what is the trend now? I foresee that S-Plus will increase its popularity in the near future...or may be already?

 

[sandeep]

As a newbie to finance, I really don't have much idea as to how much databases are used in finance. Through my previous work experience in data analytics, I can say that I used SQL as much as I could use it. It is a very powerful language designed for the retrieval and management of data and certainly not BORING ..

 

[bobbychopra]

"Readings in Database Systems, " is the best book to describe the origin of the database concept with the features that were added. It is actually a set of papers from people who researched and developed the systems and the functionality that are available in those systems.
Amazon.com: Readings in Database Systems, 4th Edition: Books: Joseph M. Hellerstein,Michael Stonebraker


SQL as the acronym implies is a Standard Query Language and is implemented by all commercial databases for our benefit. It helps us "do stuff" to data using a higher-order language as an abstraction.

Imagine the headache that you encounter when you move from one company to another [or from one system to another], which are designed to provide the exact functionality. For my case, I will assume that everyone has developed a basic calculator in the programming language of choice. Most calculators are prefix calculators but there might be others implemented as a post-fix calculator. Their apis might be different across different systems, some using 2 parameters and returning the value, others might use 2 parameters and a 3rd parameter for the return value.

What a high level abstraction would allow you to do is use the calculator without having to worry too much about how the application is implemented. In this way, SQL helps you work with data without needed to think about the underlying database or its structure.

I don't think there is too much to SQL [which is a good thing as it has managed to keep the over-increasing complexity of the underlying DB out of this realm] and it is great that you find SQL mundane and boring. Also, each database has extensions to SQL which expand on the basic usage.

Depending on your usage of DBs, you might find SQL to be sufficient or seriously deficient. There is no good support for queries that would return XML data, COBOL like keyword approach, no simplification for doing recursive or self join queries.

I hope this helps....

 

[alain]

Depending on your usage of DBs, you might find SQL to be sufficient or seriously deficient. There is no good support for queries that would return XML data, COBOL like keyword approach, no simplification for doing recursive or self join queries.

I found XML seriously deficient and overbloated :D

 

[bobbychopra]

I found XML seriously deficient and overbloated :D

It is a markup language, so I agree with you that it is excessive. It has this need of being very specific. This is the same complaint about SOAP and some choose to use POX [Plain Ol' Xml] instead of SOAP to reduce a bit of the excessiveness when using Web Services.

But there are benefits of having XML queries that are serviced directly from the DB. This can't be done for big applications but definitely for applications where the middle business layer can be pushed to the Database. This is why SQL Server 2005 is allowing web-points, service brokers and .NET stored procedures and support for XML. I guess Microsoft is betting that most applications can be changed to 2-tier using the database as the backend and the business layer and any front end which can transform the data to match your choice of presentation.


I do not mean to criticize but could you elaborate on why you find XML deficient?
-- maybe we should take this offline. By the way, just to clarify, I do not like XML too much. I have used it a lot but I hate going through pages and pages of data without a good editor capable of collapsing sections or in the absence of a stylesheet to transform the data.

 

[John]

going a bit off topic, FIX protocol messaging and FIXML (FIX based on XML vocabulary) is essential for straighthrought processing (STP) and good to know if anyone wants to get into algorithmic trading. Now we are going deep into the IT domain, so be careful ;-)

 

[DominiConnor]

SQL is useful to a quant developer, though rarely is it critical to getting a job.

There is however a signal in how much an interviewer asks you about SQL, the less quanty the job will be, and in general it will be a less good job.

 

[joe_bradley]

I can't speak as a quant, but speaking as a developer, there's something to be said about knowing SQL well enough to get the job done, just to force yourself to think differently than you would for a procedural language. For programming skill and mental flexibility, there's something to be said for knowing a few non-procedural languages well.

And pragmatically, there's nothing more annoying than having to go through the resident SQL guru to get at the data you need.

As far as XML integration, at least in my job it's a non-issue - I know MS SQL has elaborate extensions that return data formatted as XML, and I'd be shocked if other widely used databases didn't have similar facility. Yeah, XML is so verbose as to be hilarious, but if the thing you're talking to wants XML, you're better off using XML, aren't you? ;-)

 

[Anton Slepnev]

SQL is essential

SQL knowledge is merely essential for a developer who works with data and its analysis.
One may use Excel, Crystal, Busnesss Objects and other stuff, program it, and use these in their programs. But SQL and DB development at all is something different - it just brings full control over the data and the ways it is stored, searched through, transformed, analysed and, in some cases, finally presented.
As for me, SQL is one of the greatest language standards ever.

In my work as a senior developer of a front-office system I am working in proportion with 70% SQL, 20% C# and 10% C++ (older libraries). And this is a normal situation, as far as I know.

After all - learn SQL, it is very nice. You'll like the way it works with data.