August 2008 - Posts

Linq to Financial Markets : Optimizing Provider to Real-Time Quotes, Analytics, and Silverlight-WPF Visualization

Announcing a new Optimizing Linq Provider

Technorati Profile

Linq to Financial Markets

The Linq to Financial Markets provider

An easier way to consume, visualize, understand and quantify just about any information you can imagine from the world of global financial services.

  • Real-Time stock quotes to Complex Analytics of Multi-Asset Class Portfolios
  • Monte-Carlo simulation with Optional Quantitative Add-Ins
  • Efficient frontier Optimization based on Constraints you Set
  • User defined heuristics from strategic rebalancing to day-trading

Thanks to Microsoft's Linq technology (introduced in the .NET 3.5 platform), the code in the image above is 100% real. It's well established that by making things easier and more 'declarative' overall productivity goes up, quality improves and most importantly overall maintenance and future extensibility can get the attention they often lack once the non-strategic work is made unnecessary.

In the case above we get the items in our portfolio now priced (using a real-time feed) below their 90 day moving average. You can 'inject' added information as we do above for history. Another option would be to inject some 'predictive' numbers using a simulation technique.

There are many Linq providers, however most are 'technology focused' such as dealing with protocols or platform APIs. We have always been the most interested in solving the harder problems driven from the business perspective. Although there are many technologies involved here, the larger difficulty is making it available in whatever way you would like to see it, across a broad set of dimensions well beyond any technology constraint.

We support more then just 'detail' information, You can use aggregate style commands (say you want to to get you assets for a pie chart that displays asset class holdings in percent). If you then wanted to drill-down, you could use the same asset class information in a 'where' clause to limit the detail to just what you own in that class. But moving well beyond 'SQL' type logic, you can set up monitoring alerts that take action if/when a trigger is met.

The key is the same simple syntax is used for all data providers across all levels of information, including full-text scans and a new feature we're working on now for a future release incorporating an even easier natural language syntax.

This platform is not a fundamentally new way to calculate or retrieve this information. We've had access to many frameworks, API's, data providers, etc. for a long time now.

What this Linq provider does however, it change the dynamics of complexity, time to market, assumed costs in maintenance and testing, and much more. This offering's goal is to make problems in this domain become utterly trivial. Our goal is to empower your team to make trivial what was previously arduous.

You're about to 'declare' what you want, rather then describe each step in the process. If your thinking it looks a lot like 'SQL' you're 100% correct. However SQL could never achieve what Linq provides.

Comparing SQL to what Linq is capable of is like comparing your ability to easily retrieve information from accounting to easily retrieving information from 'anything that is information'.

Due to the nature of the access to this information, we are still working out the details for the open-source version we plan to offer soon. The world of the owners of this data has not caught up to the philosophy we have for transparency and shared value. Send us an email for updates or simply register for the site feed. Also if your in the industry and have an interest we are looking for innovative early adopters. Also, we'll be posting on many of the challenges we faced and a few innovative solutions we applied to the emerging and highly strategic domain of intelligent Linq Parser development.

 

Technorati Profile
Framework API Development Best Practices using C# 3.0

PART 1

image

This content assumes you have an introductory knowledge of C# 3.0 language features in .NET 3.5 and mastery of legacy C# 2.0 Generics, Generic Type Constraints, Anonymous Delegates and related material.

I use a 'pair programming' approach with continued refactoring as this is how I would discuss it if you were coding with me, with an unfortunate one-way delivery.

The Brief Strategic View

Microsoft has slowly been moving C# in a very productive direction (this is not new, as these features existed in 2.0 although not nearly as well integrated) to provide 'Functional Language' features. If you don't know or care about language semantics, just know that Linq and especially Lambada Expressions are about empowering you to use executable code like a variable, aka to leverage the power of functional programming. For more on this, read this MSDN article by Joel Pobar (former CLR Team) or read the next set of posts (part 2 onward) as I will go into the depths of this.

I think of Lambadas as an incredibly focused and powerful domain specific language for delegates.

In this sense they are quite similar to Regular Expressions in that they are really good at their focus area.

What do I mean by good?

  • Terse yet Understandable / Maintainable
  • Syntax tailored to the need, not the other way around.
  • Highly effective for problems that are orders of magnitude more difficult without them (a simple 10% improvement would not cut it)

 

 

 

Painless Intro

 

I'll start with a fairly trivial, yet important example (I use it every day). Many times when comparing Strings I want to ignore case and culture (the InvariantCulture). This is provided by an overload as such as you likely know:

[Test,Category("BaselineCore")]
public void shouldAsserStringCloneInvokeEqual() {

var baselineString = "This is a TEST CASE to IgnOrE Casing";

var stringUpper = baselineString.ToUpperInvariant();

Assert.IsTrue(baselineString.Equals(stringUpper,
                                     
StringComparison.InvariantCultureIgnoreCase));

The test results are shown above. Pass. Ok so I really like short, concise code that is understandable at a glance. Also it's a pain to always (even with ReSharper) use this (and I have seen people use RegEx for this! RegEx is awesome but overkill for this issue).

Refactoring

  • Create an extension method on String
  • Decide on a good name for the method (this is SO important and for most an afterthought!)

I've settled on calling this new method on String 'EqualsCore' as that is what we are doing, making the conditions for a match 'simpler' and seeing 'just the core values' are the same (anyway it makes sense to me)., I suppose this could be 'EqualsRelaxed' or whatever..

Here is the test case (no code yet):

[Test, Category("BaselineCore")]
public void shouldAsserStringsEqualUsingExtension() {
var baselineString = "This is a TEST CASE to IgnOrE Casing";
Assert.IsTrue(baselineString. EqualsSimple(baselineString.ToUpperInvariant())); }

Now we write the code. Here is the container for the extension method:

public static class StringExtensions {

public static bool EqualsSimple(this string sTarget, string compare) {

return sTarget.Equals(compare, StringComparison.InvariantCultureIgnoreCase); }

}

Indeed they both pass:

image25

 

 

 

 

 

Since every type inherits from Object, and Equals is defined on Object, all instances should support this approach, and I could be early bound by using Generics.... Hmm...

I tried this (note: I gave it a new new 'EqualsThis' to separate them.

public static bool EqualsThis<TTarget>(this TTarget sTarget, TTarget compare){



return sTarget.Equals(compare);


}

Functionally not that interesting at all, but a test. So I typed in the following and wow... It works from Intellisense's view... Ok it compiled! Wait.....FAIL! But why:?

OK here is the new test:

[Test, Category("BaselineCore")]

public void shouldAsserANYTHINGEqualUsingExtension() {

const String baselineString ="This is a TEST CASE to IgnOrE Casing";

var sb = new StringBuilder(baselineString);

Assert.IsTrue(sb.EqualsThis(new StringBuilder(sb.ToString())));

}

Interesting...

Here is the documentation for what Equals means by default from Microsoft:

Returns: true if objA is the same instance as objB or if both are null references or if objA.Equals(objB) returns true; otherwise, false.

So our code fails using the extension yet this returns true:

[Test] public void shouldAssertStringBuilderExplicit() {

const String baselineString = "This is a TEST CASE to IgnOrE Casing";

var sb = new StringBuilder(baselineString);

var sb2 = new StringBuilder(baselineString); Assert.IsTrue(sb.Equals(sb2));

}

So Reflector to the rescue once again. I could see in Reflector what I believed the issue was. Indeed the StringBuilder class has an overloaded Equals, and even making the extension method cast to the generic type directly was a no go.

So what do you think? Why would this compile fine with absolutely no problems (and that is correct it turns out), but FAIL at runtime on the assertion when the same line above passes? Skip ahead and reply with the answer if you know it....

This exposes one of the dangers that we must be incredibly careful with. It has always been poor design in my opinion to encourage developers to override common methods such as ToString() and Equals(object X) with their own behaviors as you force consumers of the API to understand IMPLEMENTATION. You cannot ensure your OK simply from a contract. This is known to be evil....

Of course this is a legacy style and will be slowly phased out.

Spin up reflector and look at the code for the OVERLOAD that StringBuilder has:

public bool Equals(StringBuilder sb){

if (sb == null) return false;

return (((this.Capacity == sb.Capacity) && (this.MaxCapacity == sb.MaxCapacity))
&&
this.m_StringValue.Equals((string) sb.m_StringValue));

}

Of course! How else could a StringBuilder claim to be 'Equal' to another... In fact it is perfectly reasonable but again shows the danger of late binding, making assumptions about how any 'object' type will perform.

So there was no real way for our extension to call the 'correct' equals. It called the base definition given above which is obvious now why it failed.

So how do we fix this for the general case?

Here is the test case which I got working.. If your not familiar with this style of code,

This is the foundation we build layer after layer on and illustrates the core of this post.

[Test] public void shouldAsserANYTHINGEqualUsingExtension() {

const String baselineString = "This is a TEST CASE to IgnOrE Casing";

var sb = new StringBuilder(baselineString);

var sb2 = new StringBuilder(baselineString);

Assert.IsTrue(sb.EqualsThis(x => x.Equals(sb2)));

}

Lambadas are like an incredibly focused and powerful domain specific language for delegates. In this sense they are quite similar to Regular Expressions in that they are really good (and to quantify good, I mean clear yet precise, not overly verbose yet highly effective for problems that are more difficult without them).

So what about the implementation? Here it is:

public static bool EqualsThis<TTarget>(this TTarget sTarget,

        
Predicate<TTarget> EqualsDelegate) {

                             return EqualsDelegate.Invoke(sTarget);

}

It's all about Expressions! Think of them as varied ways to receive executable code that you can 'invoke' literally, that must meet the contract you define.
This is so basic after we cover what the really useful applications are. However remember this has nothing to do with the examples, only the concepts they represent.


 
  1. Combine Generics and Generic Constraints to your Extension Methods but BE CAREFUL and ensure you are covered by unit tests
  2. Try to always think a level of abstraction above where your immediate need is to see if your solution indeed has wider and perhaps far more valuable contribution.
  3. Hide complexity behind your Framework API, and focus on crafting work that others will easily consume.
ReSharper 4.01 RC1 Released : See the Team Here as Well

Download it (or when this is outdated, later releases) from:



Build #: 922 VERSION 4.0.1 RC1

  • photo Sergey Anchipolevsky
  • photo Igor Alshannikov
  • photo Alexander Anisimov
  • photo Dmitry Avdeev
  • photo Sergey Baranov
  • photo Yury Belyaev
  • photo Natalia Belyaeva
  • photo Dave Booth
  • photo Elena Bukreeva
  • photo Jana Charif
  • photo Nikolay Chashnikov
  • photo Alexander Chernikov
  • photo Roman Chernyatchik
  • photo Sergey Coox
  • photo Sergey Dmitriev
  • photo Ilia Dumov
  • photo Michael Gerasimov
  • photo Sergey Golovachev
  • photo Alexey Gopachenko
  • photo Peter Gromov
  • photo Dmitry Jemerov
  • photo Maria Khalusova
  • photo Valentin Kipiatkov
  • photo Cyril Konopko
  • photo Anna Kozlova
  • photo Mikhail Kropotov
  • photo Alexey Kudravtsev
  • photo Olga Lobacheva
  • photo Dmitry Lomov
  • photo Vyacheslav Lukianov
  • photo Anton Makeev
  • photo Egor Malyshev
  • photo Maria Marakulina
  • photo Kirill Maximov
  • photo Sasha Maximova
  • photo Irina Megorskaya
  • photo Lucie Morawiecova
  • photo Alexander Morozov
  • photo Maxim Mossienko
  • photo Ekaterina Musienko
  • photo Ann Oreshnikova
  • photo Eugene Pasynkov
  • photo Vaclav Pech
  • photo Alexey Pegov
  • photo Marie Pejcharova
  • photo Eugene Petrenko
  • photo Irina Petrovskaya
  • photo Mikhail Pilin
  • photo Konstantin Polonsky
  • photo Julia Repina
  • photo Elizaveta Revyakina
  • photo Ilya Ryzhenkov
  • photo Andrew Serebryansky
  • photo Ilya Sergey
  • photo Maxim Shafirov
  • photo Tatiana Slavina
  • photo Olesya Smirnova
  • photo Konstantin Solomatov
  • photo Oleg Stepanov
  • photo Pavel Sher
  • photo Ekaterina Shliakhovetskaja
  • photo Oleg Shpynov
  • photo Gregory Shrago
  • photo George Udov
  • photo Sergey Vasiliev
  • photo Natalie Yaremych
  • photo Yegor Yarko
  • photo Timur Zambalayev
  • photo Sergey Zhukov
  • photo Eugene Zhuravlev
  • photo Alexander Zverev
Page view counter