I see what you're going for here. And it think its good in concept. The whole "drunk drummer" beat works great in a lot of scenarios. The issue i have here is the "pockets" you have are too deep to groove with. Try reducing the time loss on the drum loop. A good solid beat with little variation goes a long way. Groove is vital to lo-fi, if the beat doesnt have something to grab onto when its riding along the record, you loose the groove.
