I implemented a very simple SIP and RTP stack using Ruby. At first it ran terribly, but with some tweaking, and I think running a newer version of Ruby, I got it so that the sound quality was very legible. Not perfect, but good enough. Ruby certainly isn't the best language for it, but it was a good learning experiencing in understanding how the protocol works. For example, I quickly learned what the "tag" parameter does.
However, out in industry, a good rule of thumb is that if it hasn't been around for at least ten years, it's not a usable SIP stack. SIP interoperability is incredibly fragile.
The basics look easy. Here are some minimally required headers, looks a lot like HTTP, send a message, get a reply, bam, done. I remember when I first started working with SIP at age 19 or 20 (ten years ago), I had the conceit that one could just read RFC 3261 and whip up a little SIP stack real quick. "Looks pretty easy".
Little. Did. I. Know. It took me several more jobs and three or four years of consulting beyond that to get to the point where I would say I had anything remotely resembling commercially valuable SIP expertise to offer. These days I have a consulting business around that premise, but that's ten years and a whole specialist career down the road. All for a little HTTP-like protocol with human-readable headers.
There are only a handful of viable commercial and FOSS SIP stacks out there. All painstakingly evolved to get there. For any serious endeavour you should use one of them. :-)
Interesting. Sounds like you were going for quality over bandwidth efficiency. Maybe a feature request, but it'd be awesome to be able to chose G.711 or G.729 based on bandwidth requirements. :)
Just a question, not trying to degrade what you've done here which is really cool by the way.