you're opening a very complicated can of worms there with your inquiry. To be quite frankly and bearing in mind how complex the German tax system is: you should definetly discuss this with a tax advisor before doing anything (or not).
While there can be some tax advantages when marrying, you could also face other costs that have to be factored in: for instance, if one of you is privately health insured and the other with public
health insurance, the baby might have to get private health insurance too if the two of you get married whereas if the mother is with public health insurance unmarried she can get the baby at no extra costs covered with her pubic health insurance. Of course, if you have either both public or private health insurance, it does not matter.
But this is only one of the many loopholes and therefore you need to discuss this all with someone qualified, a professional.
And since I am expecting my first child as well very soon (only 5 weeks more to go!): good luck and enjoy the adventure!
Cheerio