In chronic kidney disease (CKD), the gut-microbiota metabolites indoxyl sulfate (IS) and p-cresyl sulfate (PCS) progressively accumulate due to their high albumin-binding capacity, leading to clinical complications. In a prospective crossover controlled trial, 60 patients with CKD grades 3B–4 (GFR = 21.6 ± 13.2 mL/min) were randomly assigned to two dietary regimens: (i) 3 months of free diet (FD) (FD is the diet usually used by the patient before being enrolled in the Medika study), 6 months of very low protein diet (VLPD), 3 months of FD and 6 months of Mediterranean diet (MD); (ii) 3 months of FD, 6 months of MD, 3 months of FD, and 6 months of VLPD. VLPD reduced inflammatory Proteobacteria and increased Actinobacteria phyla. MD and VLPD increased some butyrate-forming species of Lachnospiraceae, Ruminococcaceae, Prevotellaceae, Bifidobacteriaceae, and decrease the pathobionts Enterobacteriaceae. The increased level of potential anti-inflammatory Blautia and Faecalibacterium, as well as butyrate-forming Coprococcus and Roseburia species in VLPD was positively associated with dietary intakes and it was negatively correlated with IS and PCS. Compared to FD and MD, VLPD showed a lower amount of some Lactobacillus, Akkermansia, Streptococcus, and Escherichia species. MD and VLPD reduced both the total and free serum IS (MD −36%, −40% and VLPD −69%, −73%, respectively) and PCS (MD −38%, −44% and VLPD −58%, −71%, respectively) compared to FD. VLPD reduced serum D-lactate compared to MD and FD. MD and, to a greater extent, VLPD are effective in the beneficial modulation of gut microbiota, reducing IS and PCS serum levels, and restoring intestinal permeability in CKD patients.