While discussing the big problem that all of us have, Paul brought up how simply being a judge, a circumcised person, or a teacher doesn't help us in the scenario. But then Paul addresses a comment regarding people who do not believe. While the unbelief is not specified in a particular object, the unbelief is very likely unbelief in God, God's standards, God's character, or something similar.
Regarding people's unbelief in God, Paul declares that the unbelief DOES NOT make God's faithfulness useless (3:4)! Paul then wrote to "let God be true but every man a liar" (3:4). Analyzing the phrasing is a bit tricky, but here's some notes. According to the CEB, CEV, ERV, ICB, NIRV, NLV, and NLT translations, the phrasing talks about how God's words are true even when everyone on Earth is lying. Make sense, since this is how truth works. Though it can be implied that everyone has lied, especially in areas regarding God (especially to oneself).
Paul then takes a quote from Psalm 51:4 -- part of the psalm when David realized he messed up big time. The quote talks about how God "may be justified in [His] words" and "may overcome when [He] is judged". In other words, God has justification that, when logically considered, can be considered reasonable, and God can basically counter counterarguments (with a good rebuttal). David wrote that part in Psalm 51 because he realized that God's verdict on David's guilt is legit, and David testifies of this personally.
But it's worth noting that Psalm 51:4 is also part of the Psalm where David asked God to forgive him. David did so because he knew of God's great compassion and faithfulness. Going back to Romans 3:3-4, we see a similar idea that Paul conveys -- God is still faithful, even when we screwed up against God. This is part of the good news that God will expand on later in the chapter.