Mattress stitch is not a knitting stitch at all. It’s a way of sewing together two knitted pieces, using a darning needle (a thick sewing needle with a large eye) and a length of the same yarn that you used to knit the garment.
Mattress stitch is likely the most popular way to assemble something that’s been knitted in more than one piece, because it keeps the seam flat. On the right side, it’s invisible.
You can imagine how clunky and chunky a seam would be on a knitted item if it was overlaid and hemmed the way that fabrics are seamed! Knitted garments being thick, it’s important that the seams stay flat.
To attach two knitted pieces using mattress stitch, you thread your darning needle with your yarn, and take it around each of the running threads on both pieces. The running threads are the horizontal yarn that goes between the Vs (if you did knit and purl stitch).
You should go right up close to the edge of the knitted piece – that’s important so it sits flat and doesn’t overlap. Take each of the horizontal strands separately, and just take your needle over to the other side and sew them together.
Here’s a video of how to do mattress stitch: