Cleaning a soiled needlepoint is not an exact science and you might need to try a few different things.
If your needlepoint canvas looks just a little grubby try vacuuming it first. Place a light cloth over the end of the vacuum tube so you don't transfer the dirt that is there to the needlepoint. Vacuum it very lightly, holding the nozzle slightly away from the needlepoint so it doesn't get sucked into the tube.
If your needlepoint canvas is still dirty you will need to gently wash it. First, test a small piece of the canvas for color-fastness. Dyes with red in them, especially, often run. Moisten a clean white terry cloth or other absorbent towel and gently dab at the canvas. If the color does not come off on the towel then you should be okay to wash the canvas in a mild laundry detergent and water solution. There are needlepoint cleaning products on the market that can also be used. Make sure the product is free of perfumes and dyes. If the needlepoint is stitched with wool fiber then a wool wash such as Woolite is good. If a cotton or synthetic fiber then use any mild, colorless, fragrance-free detergent (but not hand soap). Make sure the solution is not too "soapy" as you do not want to leave a residue. Also make sure that the container you are handwashing your canvas in is clean.
If there is only a small area of dirt on your needlepoint canvas then try to "spot" clean it. Do not rub. Dab the soiled area with the detergent solution firmly using a clean terry cloth. You are trying to lift the dirt. When you think you have lifted all the dirt you can, get some clean water and repeat the process.
If your needlepoint is very dirty then you are better off handwashing the whole piece. Once you have gently washed it in the detergent solution place the needlepoint between two dry towels and roll it up (do not twist the needlepoint, you are just aiming to get the towels to soak up the moisture). You will then need to block the piece before it dries. Instructions for blocking needlepoint are here.
This probably calls for dry cleaning. Contact cleaners in your area, especially those advertising cleaning of furnishings, and ask them if they have experience with this type of fiber. Ask the cleaner to use new cleaning fluid and to not steam press the piece (or only do so lightly) as this may damage the fibers.
If you have an heirloom piece, or an especially precious item of needlework or tapestry, you may want to look for a place that does needlework restoration. For example, the Textile Conservation Workshop.