As Jesus confronted the Pharisees who rejected Jesus and proclaimed his link to the Father, there are a few others who decided to believe in Jesus.
Jesus then makes comments towards people who believed Jesus. It's hard to pinpoint exactly the characteristics of those who believed Jesus as written in 8:31 (especially with the future comments and actions later in John 8), but here are two ideas:
The people who believed Jesus (as written in 8:31) did not necessarily believe in Him. They likely believed that He is a prophet ... and that's it. The lack of belief helps explain verse like 8:37. Tom Constable wrote a lot more in the commentary of John 8:31 (Study Light).
Jesus did a side comment to the new believers in John 8:31-32 and then continued to the general crowd after others asked a question. The reasoning is that the comments as described in 8:37 couldn't happen for people who believed in Jesus. Bob Wilkin wrote two articles -- one older and one more recent.
Regardless of who is the recipient of Jesus's words, Jesus's words themselves are much more important. Jesus proclaimed that abiding in Jesus's word means being his disciples (8:31). Then they can know the truth that sets them free (8:32).
With the comments, Jesus proclaims the importance of abiding in Jesus's word. According to Merriam-Webster, the word "abide" can mean "to continue in a place" or "to remain stable or fixed in a state". Thus, while believing in Jesus for everlasting life is designed to be a one-shot deal (as described in John 5:24), abiding in Jesus's words is a lifestyle choice that he calls all believers to do. The lifestyle choice is part of "discipleship" -- training under the Master Trainer Jesus. As we walk in discipleship, we can know truth that sets us free, and as explained shortly later, the freedom means freedom from screw-ups. Jesus guides us not only to everlasting life, but to a present life free from our past.