God rejoices in the works of creation because they point us beyond themselves to God himself.
God means for us to be stunned and awed by his work of creation. But not for its own sake. He means for us to look at his creation and say: If the mere work of his fingers (just his fingers! Psalm 8:3) is so full of wisdom and power and grandeur and majesty and beauty, what must this God be like in himself!
These are but the backside of his glory, as it were, darkly seen through a glass. What will it be to see the Creator himself! Not his works! A billion galaxies will not satisfy the human soul. God and God alone is the soul's end. Jonathan Edwards expressed it like this:
The enjoyment of God is the only happiness with which our souls can be satisfied. To go to heaven, fully to enjoy God, is infinitely better than the most pleasant accommodations here. [These] are but shadows; but God is the substance. These are but scattered beams; but God is the sun. These are but streams; but God is the ocean.
This is why Psalm 104 (verses 31-34) comes to a close like this, with a focus on God himself. In the end it will not be the seas or the mountains or the canyons or the water spiders or the clouds or the great galaxies that fill our hearts to breaking with wonder and fill our mouths with eternal praise. It will be God.
Pleasures of God, Multnomah Books (Colorado Springs, CO), pages 94-95