Jesus made some important claims on his link to the Father as well as his authority in matters on life and judgment. He made such claims to the Jewish leaders, who made accusations to Jesus. Many of them are probably questioning the legitimacy.
In the next few verses, Jesus will point out support for his claims. First off, Jesus proclaims that if Jesus "bear[s] witness of [Himself]", then Jesus's witness "is not true" (5:31). Take note of the context -- Jesus was bringing up how in Jewish law, two witnesses are needed (which he'll later show). From Mounce's Interlinear, the word for "true" can be interpreted as "reliable", and such phrasing fits into the context. Jesus is essentially using witnesses to show that his word is reliable in the context of Jewish law.
With the establishment on how Jesus will demonstrate his reliability, Jesus tells the Jewish leadership that "there is another who bears witness of [Him]" and that Jesus knows that "the witness [about Jesus] is true." (5:32) Then Jesus goes on to show who these witnesses are.