Troop Tower adds strength also for your opponent's besiegers stacked on sites he controls.
Olog-hai of Mordor has his cost reduced also for your opponent's minions stacked on a site.
Well, I can't really find anything in rules or in game texts of these cards that would suggest otherwise. Troop Tower - "... for each besieger stacked on a site", Olog-hai of Mordor - "... for each minion stacked on a site"
During your turn, only these cards in play are active:
• sites on the adventure path,
• sites in any player's support area,
• your Free Peoples cards,
• your copy of The One Ring, and
• your opponents' Shadow cards.
All other cards in play are inactive. Inactive cards are not affected by the game and do not affect the game.
(Comprehensive Rulebook, entry on "active")
When I am FP player, my besiegers are not active and because of this don't count for my opponent's
Troop Tower.