Now let’s say you are given a vector x and y, say:

In this case the y values here represent probabilities and one thing you’ll notice is that we have probs of 1 and 0. Both of which are bad. So we apply a little “laplace smoothing” to them:

Now let’s look at what the data looks like.

Well, it may be sigmoidal, maybe not. For now let’s assume we think it is. Which we do for the most part.

Okay, now let’s solve for a line in our sigmoid function:

Now the left hand side of the equation is a line and the right hand side is some logarithm of the y data. We can plot x versus this right hand side:

Looks pretty interesting and hopefully at this point it also looks kinda linear, which it kinda does.

Now let’s fit it with a line:

Which produces this output:

We can also test the significance of the fit with an ANOVA.

Which produces this output:

And we can plot the resulting fit in linear space:

Now let’s see how our fit looks back in normal space using our formula with our derived a and b values.

Voila! We have fit a sigmoid function to our data.

http://twitter.com/caddymob

I think I’m waiting now.

UPDATE:
Jason posted details describing his troubles and how he worked around it here:
http://corneveaux.com/blog/installing-snow-leopard-a-big-fail-and-a-work-around

The columns of the data are:

• Representative Name
• State
• Number of Earmarks
• Total Cost
• Solo Earmarks
• Solo Cost

The solo columns are for earmarks where that representative was the only representative who requested the earmark.

Republicans vs. Democrats

The first obvious division is to split the data on party lines and see if their behavior is any different.

Column	Mean Democrats	Mean Republicans	P-value	Histogram
Total Earmarks	26.8	22.3	3.26E-4
Total Cost	$37,402,953	$30,683,681	0.02873
Solo Earmarks	10.3	9.2	0.0925
Solo Cost	$7,606,210	$7,746,574	0.7782

Red: Republicans, Blue: Democrats.  Significant p-values are in bold.

Table 1. The above table shows that the average number of earmarks that are approved is significantly higher for democrats than for republicans and also that democrats get significantly more money for their earmarks than republicans do.  Please note that when I say “significantly” I mean it in a statistical sense.  The p-values (or the probability that the difference between democrats and republicans is purely by chance) for the first two rows of the table are signifiant ( less than 0.05).  You can interpret this as having a less than 5% chance of this occurring completely by chance.  However, when looking at solo earmarks there is not a significant difference in the number of earmarks granted or their cost.

For the Statisticians: To calculate the p-value I used the wilcoxon rank sum test as the distributions are not normally distributed.

However, I feel obligated to point out that because the house has a majority of democrats (237 to 163) it may be easier for democrats to get their earmarks passed, thus there are more for democrats.  For comparison, data from when the GOP has control of the house is required.

http://tinyurl.com/2wjoxy

NPR recently did a story that teenagers have been using the above sound as a ring tone to evade the prying eyes ears of adults.

So we started a little investigation of our own, and we need your help! Just click on that sound file, play it, and respond in the comments with your age, gender, and if you can hear it or not.

SCIENCE!

My Trekkie heart has melted! I am in love and I won’t deny it.  I am ready to stand up on the roof tops and proclaim my love for the series that, up until last night, I believed was forever destined for re-run nostalgia.  Too much?

Too bad!  I am thrilled with the new movie.  I am head over heals for the young, sexy, talented new cast and am delighted that the writers tackled the new/old story line with a splash of nostalgia and a heap of creativity.

It’s a whole new game now, we can again visualize a Star Trek story line that isn’t confined to the plot devices that were relevant 10, 20, 30, or even 40 years ago.  I love that I don’t have to worry about how what happened in this story might affect my beloved Picard (le sigh Jean Luc).  I don’t have to worry about plot lines that no longer make sense (with the major exception of Spock Prime living out his life in a parallel universe as opposed to the universe we all know and love).  In all honesty, this is a huge relief.

Besides, this new cast brought the “sex” back to the series.  Never had I thought to think of what a restrained yet sensual kiss from a Vulcan might be like.  Thank you Zachary Quinto for making it abundantly clear that T’Pol was not the only sexy Vulcan in existence.   I am doing my best not to sound like an obsessed fan-girl, but I don’t think I am succeeding.  Forgive me men if you do not share my good opinion in the lusciousness of the Spock-Uhura kiss.

More to that point, as a woman, I always had a hard time relating to Kirk.  In TOS he was the John Wayne of the Federation.  No rules, lots of sex, and your mans-man!  But after Chris Pine’s performance, I feel like I understand Kirk better.  All of the above still holds true, but it just makes sense now.  Don’t bash me for not “getting it” sooner, I am just a little late to arrive on the Captain Kirk bandwagon.   Oh, I almost forgot to mention, the young Kirk is super sexy too.

I know that I am a minority in the Trek Universe (being young and female), so perhaps my male counter-parts won’t share the same passion that I do for this new/old series.  I know there are things to complain about when a favorite story attempts to expand in a new direction (don’t get me started on the Harry Potter movie franchise), but I think that this deserves an “A” for effort!  Keep ‘em comin’ boys.

The “old-school” informaticists have relied on Perl (yes, Perl) to develop these scripts.  But in the modern ages where languages like Python and Ruby are available to us, we don’t need to be knee deep in archaic idioms and clunky syntax.  I have been relying on Python for years now.  My implementation speed has increased substationally, I have less headaches, and life is generally better.

It turns out, however, that many of my colleagues and peers have a great appreciation for Ruby, which seems to be gaining a lot of traction lately.  As a self-named agile software developer I feel it would hypocratic if I didn’t pick up Ruby.  Afterall I need to have as many tools in my box as possible and learning Ruby will only make me a better informaticist.

My first impressions for Ruby have been very positive.  It is a truly object oriented language in ways that Python falls short.  Also I just love the dual implementation of most of the methods for when you want to modify the instance or you would rather return it.  For example, say we have a list:

The method call with the exclamation point at the end mutates the instance ‘list’ while without it, it will return a new list.  This is one of my biggest griefs with Python.  It causes code that should only take one like, take three. (Not to mention just trying to remember which way the method works)

Python

Ruby

You just can’t argue with that.  Also I found this great resource for Ruby.  If you are an experienced programmer (especially if you are familiar with high level languages) this page basically teaches you everything you need to know about Ruby to get started.

You see, what better way to take over the world than to make all peoples feel self-important and over-confident. My latest iPhone toy, LaughTrack, allows you to carry around a studio audience in your pocket. All your jokes will be laughted at. All your snide comments, joyfully giggled at. All your great accomplishments will be cheered and applauded. This makes you and everyone else feel like a star and as such you want others (namely me) to do your bidding. This will make all of you hand over enormous amounts of control over your lives to me. With this new control I will spread LaughTrack even deeper into all societies and …. I think you see where this is going.

You may think that it is foolish for me to divulge this fiendish plot to the very people from which I will extract their freedom, but I know it’s okay, because I’m getting loads of applause and cheering from the audience.

Check out more on LaughTrack or buy it from iTunes ($0.99).

What I have found most dissapointing is the interface for Copy and Paste. I loaded up the CNN front page and tried to copy a few phrases. It didn’t go so well. It keeps trying to copy the entire text. Then it displays little crop buttons, but don’t even think of trying to use them. If you drag toward the edge of the screen (like when you’d want to scroll down) they switch modes and become completely unusable.

Perhaps with some more time I would be able to figure out how to use it nicely, but this is way outside of “it just works” right now. Hopefully the Apple designers will be improving this feature before 3.0 is released to the public. Otherwise there are going to be a lot of very frustrated people out there.

Oh, and not having to enter my username/password every time I need to log into a website. Priceless.

I don’t want to give too much away, but it’s loads of fun and it’s free. Download it now and have some fun with your friends!

Free Download here.

-Spicy