Jan 17, 2013
I'm in Brasil now. I already posted a blog on my Brasil blog.
Short story time. Twitter doesn't work in the subway and I didn't want to write interesting stuff to Twitter after it ate a message, so here are a few notes about Tuesday night in New York.
Read more »
Jan 12, 2013
I've been blogging more than usual since I released AI3 on Christmas Eve. You should check it out. In comparison to all websites I have released, AI3 has the most potential and should get the most respect. I purchased a super-fast server (SSD especially for fast database lookups), leased a super-fast colo space for it, and am going to add to it regularly. As a feature of AI3, I will attempt to keep a regular blog here with insight into what I think about each feature of the website is and then I will make a page with that data on ai3 using a simple slug. I've already done a few if you want to look at the past few blog posts.
The feature that I'm going to discuss today is single-minded research of a single difficult topic. Searching for a common word in Google can be one of the most frustrating things in the world. What you really want is for someone to answer the question you are asking, not learn every way to misunderstand what you are asking. Sometimes AI3 will fail, there's no doubt that Google is more in depth than anything I can create even if I had all of Wikipedia. So let's get in depth on a very simple question. It's not one of the easy questions I've been dealing with. Let's ask: "Is the word 'We' used more positively or negatively?" By that, I mean "Is the sentence 'We plan to solve poverty by 2017,' more common than 'We can not solve poverty by 2017'?" But not just that sentence, but every sentence which is in the positive "We *verb*" vs "We *verb* not". This is a deviously difficult problem. Even with a huge corpus, definitive answers require statistical analysis of a ton of stuff. Let's attempt it though. Start with We and we. All words in AI3 are case-sensitive, which is why there are links to all variants of we on the We word page. 1276 pages is too many unless we have a script. Let's try collocation of We. It's a slow process because We is such a common word. You can look below if you're impatient. While you're waiting, maybe try looking at a few sentences. The second sentence is:
`` We didn't want town work '', Jones said.Eureka already? Yup. All we need to do is find similar words on We and every word that is in the negative. That's pretty easy, right? There are only four pages of words that contain n't and most of them are pretty uncommon. Note that there's a bug where dashes assume that two words are one. That's a problem with my parser which should be more intelligent about whitespace. So manually or automatically, we can start searching for sentences that contain We didn't and so on. Since the related page doesn't have a count (due to slowness), we are stuck just trying a high page number and using a binary search from there. If you don't know what a binary search is, let me explain. Let's say that there could be upwards of 100 pages of sentences or more. Simply skip to page 100. If it gives you an error, then there aren't that many pages. Go to half that number, page 50. Half the number again and again until you come up a valid page. Then pick a number half way between the valid page and the invalid page. After a few hits, you will find that page 6 is the end of We didn't. In total, it should only take 7 tries to find any number between 1 and 100 because 2^7 is 128. If you don't understand the math, hopefully you'll understand the process. Anyway, now we have a way of counting all the negative sentences. Then we simply need to count all the sentences that contain We. That can be found on the We word page. But let's say that you thought this algorithm through and have some skill with a database. How long would it take you to come up with the solution? Read more »
Jan 2, 2013
AltSci will soon have a proper server with a lot of bandwidth. I just bought a $1000 server off Newegg and it's going to be fast. Intel Xeon E3-1230 V2 Ivy Bridge 3.3GHz (3.7GHz Turbo) 8 threads, 16 GB ECC RAM, 240GB SSD. I'm going to send it to a datacenter with a lot of bandwidth to do some fun projects.
As I spend less and less time working on my new website AI3, I have to choose the most important features that I want to work on. Completeness is important (the page on Emma Goldman is missing everything after the header), social features are important (users aren't implemented yet), UI is important, and adding more data sources is important to my users (Creative Commons blogs will be added as time goes on), but more important to me is functionality.
Read more »
Dec 27, 2012
First of all, check out AI3 because that's really important and fun. I've added two features. See if you can find them. Google has started indexing me slowly. I hope they don't spend too much time before they decide that it's not worth a high ranking. CPU time isn't free. I've been slowly promoting words and sentences that I like. You can tell that my favorite vocabulary is very biased toward things that I think about but don't get the opportunity to speak about. That will change as my power of linguistics improves.
Now for the music, 2012 December 27 [flac]. I actually made this today in a few hours. Don't expect too much from it, it's an experimental 3 instrument song, (plucked instrument, bass, and electric snare) played on keyboard with Rosegarden, ZynAddSubFX, and JACK. It shows the progress of my musical composing skill, still quite amateur. The plucked instrument (aka guitar but it wasn't really a guitar so much) is by far the best part of the song. I rerecorded the first 4 seconds because the original patch (which is quite good in the middle and end) started off with a cacophony of horrible discord. You might say that the entire thing is a cacophony of discord but I rather like it. The bass was put in when decided that the plucked instrument had too many silences that I didn't feel like removing. So I figured out how to make the bass work and then just hit 5 keys over and over again in any rhythm and melody that made sense. It doesn't sound very good at high volume, so I took it to its lowest audible volume so that it'd only pick up the silences and a bit the rest of the time. The drums were quite easy copy and paste and took almost no time to work with. I tried to do a really fast paced drumming session and it totally didn't work at all with the slow plucked instrument, so I chucked it. The end is rather abrupt because it ends on a short high note preceded by another high note instead of a long low note. I left it that way because endings are overrated and this is a quick experimental song and should sound like it. The middle contains quite a few silences which don't make sense and you get a little bit of the bass and snare and then a few more notes. Once I have more practice, I will certainly make my songs a lot cleaner. I wonder how many rests a rock song can get away with. Maybe 2 if it's long. Also note that the entire song is in the key of C. I haven't figured out keys well enough to use any other key than Major C so it sounds like I don't know anything about music. Well, I don't. I have spent maybe 6 months of free time in my entire life working on music and this is the output. Half of that was practicing keyboard at age 10 or so. Some of it was practicing drums with RockBand (1, 2, 3, and The Beatles) which I didn't use at all because this song didn't seem to call for it. I spent a few months of free time in 2003-2004 working on CSound, SoundTracker and MusE. Skills you learn in one music editor transfer fortunately. In CSound I created a ton of original samples which I didn't use in this because ZynAddSubFX has completely changed the way that I deal with instruments. If you're interested, there are other samples of my music here on this blog:
