Bradford Walker recently penned an article titled "You Suck at Using Lore in Your Games." In it, he excoriates infodumps and dwelling on the background the GM created:
Let me tell a truth born of 36 years in gaming: common gamers don't give a fuck about lore.
They don't care because those responsible for communicating relevant information to the player(s) routinely fail to do so, and then get surprised when said player(s) get mad at missing information that suddenly is relevant to their situation. Why? Because they tuned out the GM while he droned on about something instead of properly weaving it into playing the actual game. Why? Because they skipped the cutscene where that got mentioned because it took the player out of the game.
and later...
Tabletop has access to this also, and due to its leaning on liminality there is one other thing you can do by properly employing lore and that's to procedurally-generate brand new content. The TSR editions of D&D lean heavy on this by way of their many random content generation tables, and Classic Traveller is in the same vein. Take something you find to a Sage NPC and run with what you get; the lore directly creates new gameplay content for player(s) to enjoy- something no videogame has yet achieved, no matter how hard they try.
(Which is really sad; all of the components are there, but no one's tried to put it all together yet.)
It's one of the reasons I love reading Jeffro Johnson's AD&D session reports. He gets it, and he's showing by example that--despite protests to the contrary--AD&D is best run with this sort of thing in mind. When I get around to running New Model Colony again, I'll keep his campaign in mind. I'm getting similar vibes from Daddy Warpig's Classic Traveller game.
And it's how to make the most out of what tabletop RPGs are good at: exploiting liminality.
Make your lore relevant to immediate gameplay situations. Make it bite-sized chunks to assimilate. Make it something every player (in multi-player games) pays attention to by not letting (or pushing) one player to deal with it all.
I've pointed out before how the randomization of reaction rolls, of random dungeon content for an on-the-fly dungeon, and wandering monster checks can provide all sorts of fun - such as the orcs in the first level worshipping the party as the "keepers of the candle", or "how do we keep this goelm powered up."
Or an insane equilibrium-style gun-fu elf with aeronite pistols (yes, this is Dwimmermount, they have blasters).
Sure - there has to be something to hold it all together, but for the most part, it can very well be generated on the fly. A GM can cook up a masic map of the area and some basic opportunities, draw up a couple rough sketches of possible delves, and let the dice fall where they may.
Here's the thing: you only have to give the players relevant lore, wth relevant defined as "will any decision they make be impacted by knowing this tidbit about the world that isn't already a given from preevious gameplay?" Think of it as the little book that was used as a guide for various TV series, that defined the characters and assumptions of the story. The writer may need to know it, so that he doesn't cause continuity issues, but he doesn't have to explain it all to the audience at one time. In bits and pieces, as Bradford notes, is just fine.
Treat it as a loose guide - if you realize it would be far better if some shadowy crime gang were involved instead of royal agents... and can make it fit... go for it.
Take a look at the image at the article header. It's possibly one of the most iconic spaceships in SF roleplaying, second only to the S-type scout courier within most Traveller campaigns. Along with it, even back in CT, there was a mess of fluff developed around the Third Imperium.
It's not needed. Take a look at what's been done using the mongoose-inspired Cepheus Engine, and Hostile.
HOSTILE is a gritty, near future roleplaying setting for the 2D6 Cepheus Engine rules that is inspired by movies like Outland and Alien. It is a universe of mining installations, harsh moons, industrial facilities, hostile planets and brutal, utilitarian spacecraft.
You don't need the Third Imperium. Sure, the assumptions baked-in restrict communication speeds, allowing for more independence of action, and would come up with some form of decentralized system if not an outright imperial heirarchy, but the environment could just as easily be a bunch of megacorps and secret societies with overlapping and competing spheres of interest, with "nobility" being the equivalent of being the board member of the Weyland-Yutani corporation (or a family member).
Incidentally - this also, as I mentioned earlier, applies to storytelling.