I'm a Liftafarian. I'm a consultant and I use Lift, Scala during the day, and Go during nights and weekends.
The Go compiler is really fast, and I got so used to it that when compiling OwlCrawler was taking about 8 seconds I started to worry (granted I generate 3 executable files).
Some google-fu pointed me to adding the
-x parameter to
go build to see more details on what was going on.
Last week I woke up to find an email from Iron.io telling me that I was running low on my available IronMQ quota. Somehow I had made around 800,000 API requests in a week by running my OwlCrawler project.
The reason was easy to spot, every time we got a resource offer from the Mesos master, the scheduler was checking two message queues for new tasks. One queue is to see if we needed to fetch html from a new page, and the other queue is to see if we needed to extract links and text from the html already fetched. This resulted in making around 160,000 API requests per day, just to see if there was any work to do.
As I continue working on OwlCrawler, I have refactored the code a lot. After getting some great feedback on the Mesos mailing list, I went ahead and split the single executor into two separate
Executors that are triggered by the main
On the scheduler, I now have these two
OwlCrawler is a simple yet distributed web crawler I'm working onto learn Mesos. I'm using the mesos go binding to write the logic and use Iron MQ to trigger tasks that fetch pages. At this point I'm storing the url and html in etcd but I have plans to move that to CouchDB in the near future.
Over the past 24 hours I made many changes, but one that I specially liked was that I could pass a function as a parameter to another function. This is possible in Go because functions are first class (values? | elements?) And this made the code a lot cleaner.
Initially I only had this one function to fetch the list of flows:
While on one side Go-Cortex let's me control an Arduino using voice recognition, I also wanted to expand Cortex into helping me and my team at Brokc Alloy. we currently use flowdock for internal conversations and there were two things that I wasn't too happy about:
There were times when people would say the current temperature where they live, and that meant I had to go and convert them from Celsius to Fahrenheit.
The other case was that someone would say "Look at #44 and see if you can fix it", referring to a github issue. Again, I would had to go to the github page and add that issue number to a url, etc.
As I continue adding features to Cortex I needed to connect to the streaming api that Flowdock provides.
A quick Google search took me to this post which shows how to do that. It works, but while I was reading the code, it looked pretty low level for my taste. So I decided to try and use the http package.
This turned out not to work so well. The card has two built in microphones, but I had to be really close to the board to get a good quality recording. And even after applying some noise removal, the service I was using for voice recognition wasn't getting what I was saying (at least not all the time).
Last week I released a small project I was working on. After I published it, I kept thinking of different ways to use it and how to make it better. I wasn't too happy with the idea that I had to type my message on the browser, so Cortex could process it. My goal is to have Cortex run on several raspberry pi computers all across my house, and somehow have them all waiting for commands, but I didn't want to have to manually interact with them.
I wrote a follow up post where I added voice recording capabilities.
Cortex is a service written in the Go language that listens for regular sentences and tries to convert them into commands to execute. And in the near future, it will also give you relevant answers.
This is a sample json response:
A couple of weeks ago I wrote about using Go with Twitter Flight to write web applications.
But I didn't put too much emphasis on the go code, it was mostly to highlight some patterns which I think can be applied to any web application.
This post is more about the go code.
I'm very grateful that I always get a chance to work with very smart people, and I get the chance to learn new tools, languages, techniques, etc. Here I'll talk about Grunt, which Tim Nelson introduced me to. And I'll talk a bit about Bower and how they can work together to help you in your Lift applications.
I was going to start this post by pointing out all the bad things about Scala and how great Go is, but I decided against it, Go is such a great language, that you don't need to trash another language to point out Go's strengths. So here, I'm going to list how awesome a language Go is.
The last couple of weeks I have been using the Go language and twitter flight for a small project. I remember watching an introductory video on go right around the time I go into Scala. It look like an interesting language, I really liked their concurrent built in approach, among other features, but it was way too new as a language for me to get into.
Now, some 4 years later, they have already released their version 1.2, there are a number of companies using it in production and having good results with it and the tooling around it is pretty good.
About two weeks ago I was asked if I could review a book about Twitter Flight. That email came in at the right time, as I was looking into trying some new language and/or framework and Flight was something I had in my list of frameworks to try out.
The publishers gave me a free copy of Getting Started with Twitter Flight, written by Tom Hamshere and off I went to read it.
The book has a nice balance between showing you the basics of the framework and giving you some guidance based on the author's experience using Flight.
You can see older posts on the archive