|
|
I dug this little gem out of the archives. Enjoy!
Originally posted on paul.vox.com
In the beginning I used elm to read my mail. This was somewhat radical, especially as I worked with the team that created POPMail for the mac and Minuet for the PC, and everyone else moved to pine. Then came Mutt -- happy days -- I was able to slice and dice email with amazing speed.
A couple of years ago I converted over to Mail.app -- mostly because of the contacts and calendar integrations, and the fact that I could merge personal email and corp email accounts. In the intervening time I had to move to comcast, which meant running my own imap server proved more difficult than it was worth, so I moved to Google Apps for Your Domain, all of a sudden my personal domain is running Gmail, and I discovered it has key bindings.
All of a sudden it's mutt deja-vu. navigation with vi j/k keys? yes. Single window view (inbox/message)? yes again. Tagging messages? yes. Blazingly fast? you bet. The only thing I miss is keystroke filtering of messages.
That's one reason why I see things like Google Wave working out so well, I might be late to the gmail party, but plenty of folks have been using this as their primary mode of communication for a long long time.
Originally posted on paul.vox.com
Using an SSL Accelerator like a Netscaler is really useful, you can offload a lot of work to a device that supports this in hardware and can use SSL session affinity to send requests to the same backend. In the simplest setup the SSL Accelerator accepts the request and proxies it to your internal set of hosts running on port 80.
However, code that generates redirects and URLs works poorly because the servletRequest.getScheme(), getSecure() and getServerPort() will return http/false/80 for SSL and non-SSL connections.
One way to solve this is listen on multiple ports. Create a Connection on 80 and 443, but do not run SSL on either port. Then for the 443 port you configure it with secure="true" and scheme="https". This is suboptimal however as then you have to manage yet another server pool in your load balancer and you end up sending twice the health checks. Not so good.
You might try to solve this by using a ServletFilter. You can use an HttpServletRequestWrapper instance to change the scheme/port/and secure flag. Sadly this doesn't work, because of the way tomcat implements HttpServletResponse, it uses the original request object to ascertain the scheme/secure flag/port. Overriding these will allow application logic to see the updated values. You get into trouble when you call encodeRedirectURL() or sendRedirect() with non-absolute URLs.
Lucky for us Tomcat supports a way to inject code into the connection handling phase via Valves. A valve can query and alter the Catalina and Coyote request objects before the first filter is run.
To make your Valve work you'll need to configure your load balancer to send a special header when SSL is in use. On the Netscaler this can be done by setting owa_support on. With that enabled the http header Front-End-Https: On is sent for requests that use SSL.
Once we have these pieces in place the Valve is fairly straightforward:
import java.io.IOException; import javax.servlet.ServletException; import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase; public class NetscalerSSLValve extends ValveBase { @Override
public void invoke(Request req, Response resp) throws IOException, ServletException {
if ("On".equals(req.getHeader("Front-End-Https"))) {
req.setSecure(true);
req.getCoyoteRequest().scheme().setString("https");
req.getCoyoteRequest().setServerPort(443);
}
if ( getNext() != null ) {
getNext().invoke(req, resp);
}
}
}
Compile this, stick it in the tomcat lib directory, add an entry in your server.xml and away you go.
Originally posted on paul.vox.com Tue, Jul. 15th, 2008, 12:04 pm
tuesday
today is tuesday. Ain't it grand...
"Two Characters Sets? Seems like plenty!"
So I've been pushing Java to it's limits lately and finding some real nasty concurrency issues inside the JRE code itself. Here's one particulary ugly one -- we had 700 threads stuck here:
java.lang.Thread.State: BLOCKED (on object monitor) at sun.nio.cs.FastCharsetProvider.charsetForName(FastCharsetProvider.java:118) - waiting to lock <0x00002aab4cdf91b8> (a sun.nio.cs.StandardCharsets) at java.nio.charset.Charset.lookup2(Charset.java:450) at java.nio.charset.Charset.lookup(Charset.java:438) at java.nio.charset.Charset.isSupported(Charset.java:480) at java.lang.StringCoding.lookupCharset(StringCoding.java:85) at java.lang.StringCoding.decode(StringCoding.java:165) at java.lang.String.<init>(String.java:516)
Digging deeper we find the lookupCharset is called all over the place. The app in question is functions as a web proxy, so it's constantly reading and writing data from web pages in a variety of character sets. The method charsetForName() uses a synchronized data structure to lookup defined character sets. (Yay serialized access....)
But wait, lookup and lookup2 provide us with a cache so we can avoid the big bad synchronized method.. Sigh, here's the implementation:
private static Charset lookup(String charsetName) { if (charsetName == null) throw new IllegalArgumentException("Null charset name"); Object[] a; if ((a = cache1) != null && charsetName.equals(a[0])) return (Charset)a[1]; // We expect most programs to use one Charset repeatedly. // We convey a hint to this effect to the VM by putting the // level 1 cache miss code in a separate method. return lookup2(charsetName); } private static Charset lookup2(String charsetName) { Object[] a; if ((a = cache2) != null && charsetName.equals(a[0])) { cache2 = cache1; cache1 = a; return (Charset)a[1]; } Charset cs; if ((cs = standardProvider.charsetForName(charsetName)) != null || (cs = lookupExtendedCharset(charsetName)) != null || (cs = lookupViaProviders(charsetName)) != null) { cache(charsetName, cs); return cs; } /* Only need to check the name if we didn't find a charset for it */ checkName(charsetName); return null; }
Yes, a whopping 2-entry cache!!
Also, the keys used are not canonical, so if my app asks for "UTF-8", "utf-8", and "ISO-8859-1" with regularity this 2 entry cache is worthless, every call ends up blocking in the evil thread-synchronized data structure.
Someone send them a copy of the ConcurrentHashMap doc. please.
</rant> ....
Originally posted on paul.vox.com
At hi5 we've been busy busy busy getting OpenSocial up and running. We released our developer sandbox, and are rapidly implementing features. So check out the following URLs
Also, here's a copy of my response to Tim O'Reilly's blog post:
Hi folks, Good comments all around. However I'd like to posit that data access
is _not_ the problem. We've had universal standards for years now with
little uptake. Tribe.net, Typepad, LiveJournal and others have
supported FOAF for many, many years, which encompasses the OpenSocial
Person and Friends APIs. Not much has come of that -- there isn't a
large enough base there to get people interested. Now you have a broad industry consensus on a single way to provide
all of the above plus activity stream data. You have a rich client
platform that allows you to crack open that data and use it in
interesting ways, and finally you have a common standard for social
networks to interact with each other based on the REST api. So Patrick's statement at the Web 2.0 Expo is correct, a app running
inside a container only allows you to see what that container shows
you. However that does not mean that a container could not contain
friend references to external social networks via it's own federation
mechanism. Movable Type 4.0 has shown that you can support any OpenID
login in a single system, there's no reason to believe that social
networks could not leverage OAuth to do the same. And here's a final point to consider -- you have Myspace opening up
to developers. That's huge. That alone is going to draw more developer
attention to this problem than much of the oh-so academic discussions
of the past few years. I suggest people that _want_ OpenSocial to solve all the social
graph ills get involved on the API mailing list and make sure that
those elements are addressed as OpenSocial evolves. There's a tremendous amount of momentum. Let's not waste this chance.
Originally posted on paul.vox.com
This has got to be a bug....
Dear Amazon.com Customer,
We've noticed that customers who have purchased or rated White Noise Critical: Text and Criticism (Viking Critical Library) by Don DeLillo have also purchased Caught in the Machinery: Workplace Accidents and Injured Workers in Nineteenth-Century Britain by Jamie Bronstein. For this reason, you might like to know that Caught in the Machinery: Workplace Accidents and Injured Workers in Nineteenth-Century Britain will be released on October 10, 2007. You can pre-order yours by following the link below.
Caught in the Machinery: Workplace Accidents and Injured Workers in Nineteenth-Century Britain
Jamie Bronstein
Price: $55.00
Release Date: October 10, 2007
Originally posted on paul.vox.com
So we use this toolkit from Octazen to scrape contact lists off of various sites. Our ever eager users (ab)used this feature so much that hotmail blocked us. So I waded through reams of API docs over at http://dev.live.com and finally came up with this prototype perl script to talk to their API servers. Gets the job done for now. Will want to rewrite it in native Java and add decent error handling soon. Hopefully this post here will help other folks needing to talk to Live API:
#!/usr/bin/perl
#
# Output tab separated values for a given hotmail username/password
#
# Implementation of Windows Live Contacts API
# http://msdn2.microsoft.com/en-us/library/bb463974.aspx
#
# Uses RPS authentication described here:
# http://msdn2.microsoft.com/en-us/library/bb447721.aspx
#
use HTTP::Request;
use LWP::UserAgent;
my $username = shift || die "Need a username\n";
my $password = shift || die "Need a password\n";
my $apikey = 'YOURAPIKEY';
my $ua = LWP::UserAgent->new;
my $uri = 'https://dev.login.live.com/wstlogin.srf';
my $req = HTTP::Request->new(POST => $uri);
$req->content_type('application/soap+xml');
my $xml = <<EOF
<s:Envelope
xmlns:s = "http://www.w3.org/2003/05/soap-envelope"
xmlns:wsse = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
xmlns:saml = "urn:oasis:names:tc:SAML:1.0:assertion"
xmlns:wsp = "http://schemas.xmlsoap.org/ws/2004/09/policy"
xmlns:wsu = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
xmlns:wsa = "http://www.w3.org/2005/08/addressing"
xmlns:wssc = "http://schemas.xmlsoap.org/ws/2005/02/sc"
xmlns:wst = "http://schemas.xmlsoap.org/ws/2005/02/trust">
<s:Header>
<wlid:ClientInfo xmlns:wlid = "http://schemas.microsoft.com/wlid">
<wlid:ApplicationID>$apikey</wlid:ApplicationID>
</wlid:ClientInfo>
<wsa:Action s:mustUnderstand = "1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</wsa:Action>
<wsa:To s:mustUnderstand = "1">https://dev.login.live.com/wstlogin.srf</wsa:To>
<wsse:Security>
<wsse:UsernameToken wsu:Id = "user">
<wsse:Username>$username</wsse:Username>
<wsse:Password>$password</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
</s:Header>
<s:Body>
<wst:RequestSecurityToken Id = "RST0">
<wst:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</wst:RequestType>
<wsp:AppliesTo>
<wsa:EndpointReference>
<wsa:Address>http://live.com</wsa:Address>
</wsa:EndpointReference>
</wsp:AppliesTo>
<wsp:PolicyReference URI = "MBI"></wsp:PolicyReference>
</wst:RequestSecurityToken>
</s:Body>
</s:Envelope>
EOF
;
$req->content_length(length $xml);
$req->content($xml);
my $res = $ua->request($req);
# Ugly way of hacking out the binarysecuritytoken
my $resultxml = $res->content();
$resultxml =~ m,<wsse:BinarySecurityToken[^>]*>(.*)</wsse:BinarySecurityToken>,si;
my $binarytoken = $1;
# Request contacts
my $contactsurl = "https://cumulus.services.live.com/$username/LiveContacts/Contacts";
my $authheader = 'WLID1.0 t="' . $binarytoken . '"';
my $contactsreq = HTTP::Request->new(GET => $contactsurl, ['Authorization' => $authheader]);
my $contactres = $ua->request($contactsreq);
my $contactxml = $contactres->content();
use XML::Simple;
my $result = XMLin($contactxml, 'ForceArray' => ['Email', 'Contact']);
# parse emails
foreach my $c (@{$result->{'Contact'}}) {
my $fname = $c->{Profiles}->{Personal}->{FirstName};
my $lname = $c->{Profiles}->{Personal}->{LastName};
foreach my $a (@{$c->{Emails}->{Email}}) {
print "$fname\t$lname\t" . $a->{Address} . "\n";
}
}
Originally posted on paul.vox.com
Meraki is building a free mesh network in San Francisco. This is probably the best hope for getting this type of service in the city now that the Google/Earthlink deal fell apart. Join up! Go to http://sf.meraki.net and help build the network. When the router comes in I'll have 7th and Howard covered with 1Mbps of donated bandwidth.
Originally posted on paul.vox.com
I'm happy to announce that Hi5 has Widget support. Yes, I know that this is soooo last year. However there's a twist that makes it better. We worked closely with Rock You and Slide to integrate tightly with our site, using open standards wherever possible. For example, for slideshows we created Atom Feeds for each photo album, and a feed-of-albums feed for the list of all albums. And when it came time to share profile information for horoscopes (birthday) and languages spoken we used FOAF. Thus we get partners to adopt open standards, plus the work we did for them is usable by everyone. The only tricky part was authentication and authorization. Right now it's using our own AuthToken implementation, but it could probably be done in a better way. I looked into OpenID as a mechanism, but's way too end-user centric for this type of thing. Coming soon we should have full Atom endpoints (both in/out with WSSE auth), OpenID provider, and a few other standards based things like XMPP vCard support. All of this is being done with an Web Services Aspect Oriented toolkit called Enunciate, which has made writing these services a very enjoyable experience.
Originally posted on paul.vox.com
Earthquake in Peru, logins drop immediately. Hope everyone is safe....
Originally posted on paul.vox.com
Earthquake in Peru, logins drop immediately. Hope everyone is safe....
Originally posted on paul.vox.com
So I've spent a good chunk of today defending Six Apart from the cheap shots being leveled at them today. I won't link to them, they don't deserve the pagerank. I'm particularly angered at the audacity of the bald-faced lies in some comments. I may not be employed at Six Apart today, but I put my heart and soul into building it. I won't let a bunch of hacks harm the people still there. So, if you see anyone anywhere putting the hurt on Six Apart let me know. I'll use discourse, reason and wit to set the record straight.
Originally posted on paul.vox.com
Just caught up 10 days worth of Neighborhood posts. I now have Vox fatigue combined with Vox guilt. I didn't even read comments, for shame :( After this post I'll need to check on the 'ol LiveJournal Friends page. Don't even ask about the umpteem BlogLines blogs stuck at 200 posts... Hi5 has a new Skins system that actually can make profile pages look good. I had some input early on and made sure Vox and the SixApart styles were part of the inspiration. It's coming out really well and we've received over 200 submissions. Check out the snazzy new profile page? Designers can check out the specs page. Embeds are evil. They mess up divs and tables and are often pasted in haphazardly. Amit came up with an amazing solution. Use JTidy to clean up the user submitted content. Tags match and broken html goes bye-bye! Now back to the super-secret Hi5 Project Funk.
Originally posted on paul.vox.com
Like many others (and Vox/LJ itself) Hi5 was affected by the power outage in Colo 4 in 365 Main. We blogged about it over at the Hi5 Blog. Originally posted on paul.vox.com
We're living on the edge over here at Hi5. Our new Movable Type 4 base blog is now available at http://www.hi5networks.com/blog/ The whole company is getting involved and you'll see plenty of interesting information to come. Also, from a technical standpoint, MT4 has proved a winner. The memcached support in Data::ObjectDriver means that we can run via plain CGI, saving a bunch of time and effort to get this up and going. We should have 3-4 posts per week. Sadly I didn't get a chance to finish implementing userpics for MT4, but that should come shortly.
Originally posted on paul.vox.com
We had a great turnout at the latest PostgreSQL users group meetup -- around 35 people showed. (Oh and not the group of stylish "Hi5 folk" you see to the right :) Ram and I went over the PostgreSQL based DB architecture we use at Hi5 after the obligatory pizza feed. Quite an interesting crowd, some newbies, and some old hands. My best line of the night was in response to a question asking us when we were going to use a specific feature -- my answer was that there were more people in the room than there were employees at Hi5. :) The complete presentation is online for the curious.
Originally posted on paul.vox.com
I see that Hi5 made the list of Privacy International as posing a substantial threat to users' privacy. I find that their methodology is extremely suspect. I can't spot any consistency in the way they treat sites.
These guys dinged us because our point of contact for Privacy is our legal counsel. He is, but he's also the guy calling Malaysia at 3AM to get phishing sites shut down. We do a lot around here.
Also, these guys claim they had a pop-up advertisement show up when they clicked on the privacy page. I know for a fact that this is not possible. No advertising code is used on those pages, never has, never will. These idiots must have had some kind of malware installed to cause that to occur.
In any case, we'll let Google and them fight it out. We don't need validation from some poor excuse for a privacy group. We protect our users and give them the tools to protect their privacy.
Originally posted on paul.vox.com |
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
lindner's journal