Lightmapping doesn't work

When I made a test scene, and added in some point light and lightmapped it, then deleted the light it worked fine. The shadows where textured on.

But I have another scene, where I'm trying to bake the shadows into the house, and its not working. I bake for like an hour, then delete the light and its all black. As if nothing was textured. But in the maps section it looks as if it was properly baked.

UPDATE: So I lightmapped some blocks, in the house when the ambient light is up. Then turned the ambient light up a bit, and you can see the walls are not light mapped even though the block is....

FYI everything is set to static in both the test scene and normal scene im using.