On 02.12.2018 21:42, Lukasz Majewski wrote: > This patch provides the code to calibrate the DDR's > DQS to DQ signals (RDLVL). > > It is based on: > VFxxx Controller Reference Manual, Rev. 0, 10/2016, page 1600 > 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode" > > and NXP's community thread: > "Vybrid: About DDR leveling feature on DDRMC." > https://community.nxp.com/thread/395323 > > Signed-off-by: Lukasz Majewski <lu...@denx.de>
Thanks for looking into this! We actually tried to use the official DDRV tool, but were not able to get really good results with it! In a quick test the calibration worked here on a Colibri VF50. Will do some more testing later this week. Some minor things below: > --- > > arch/arm/mach-imx/Kconfig | 8 + > arch/arm/mach-imx/Makefile | 1 + > arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 > ++++++++++++++++++++++++++++ > arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ > 4 files changed, 404 insertions(+) > create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c > create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h > > diff --git a/arch/arm/mach-imx/Kconfig b/arch/arm/mach-imx/Kconfig > index a1566cc2ad..ca10ba683f 100644 > --- a/arch/arm/mach-imx/Kconfig > +++ b/arch/arm/mach-imx/Kconfig > @@ -78,3 +78,11 @@ config NXP_BOARD_REVISION > NXP boards based on i.MX6/7 contain the board revision information > stored in the fuses. Select this option if you want to be able to > retrieve the board revision information. > + > +config DDRMC_VF610_CALIBRATION > + bool "Enable DDRMC (DDR3) on-chip calibration" > + depends on ARCH_VF610 > + help > + Vybrid (vf610) SoC provides some on-chip facility to tune the DDR3 > + memory parameters. Select this option if you want to calculate them > + at boot time. > diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile > index 53d9e5f42b..4a07b1ea69 100644 > --- a/arch/arm/mach-imx/Makefile > +++ b/arch/arm/mach-imx/Makefile > @@ -51,6 +51,7 @@ obj-$(CONFIG_SECURE_BOOT) += hab.o > endif > ifeq ($(SOC),$(filter $(SOC),vf610)) > obj-y += ddrmc-vf610.o > +obj-$(CONFIG_DDRMC_VF610_CALIBRATION) += ddrmc-vf610-calibration.o > endif > ifneq ($(CONFIG_SPL_BUILD),y) > obj-$(CONFIG_CMD_BMODE) += cmd_bmode.o > diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.c > b/arch/arm/mach-imx/ddrmc-vf610-calibration.c > new file mode 100644 > index 0000000000..6ed3f5375b > --- /dev/null > +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.c > @@ -0,0 +1,336 @@ > +// SPDX-License-Identifier: GPL-2.0+ > +/* > + * ddrmc DDR3 calibration code for NXP's VF610 > + * > + * Copyright (C) 2018 DENX Software Engineering > + * Lukasz Majewski, DENX Software Engineering, lu...@denx.de > + * > + */ > +/* #define DEBUG */ > +#include <common.h> > +#include <asm/io.h> > +#include <asm/arch/imx-regs.h> > +#include <linux/bitmap.h> > + > +#include "ddrmc-vf610-calibration.h" > + > +/* > + * Documents: > + * > + * [1] "Vybrid: About DDR leveling feature on DDRMC." > + * https://community.nxp.com/thread/395323 > + * > + * [2] VFxxx Controller Reference Manual, Rev. 0, 10/2016 > + * > + * > + * NOTE > + * ==== > + * > + * NXP recommends setting 'fixed' parameters instead of performing the > + * training at each boot. Can you also add this information to the Kconfig? That is what people usually read to decide whether to enable an option. > + * > + * Use those functions to determine those values on new HW and write > + * default values to registers. I guess the idea is to add it back to the board specific DDR settings? Can you be a bit more clear, e.g. Use those functions to determine those values on new HW, read the calculated value from registers and add them to the board specific struct ddrmc_cr_setting. > + * > + * SW leveling supported operations - CR93[SW_LVL_MODE]: > + * > + * - 0x0 (b'00) - No leveling > + * > + * - 0x1 (b'01) - WRLVL_DL_X - It is not recommended to perform this tuning > + * on HW designs utilizing non-flyback topology > + * (Single DDR3 with x16). > + * Instead the WRLVL_DL_0/1 fields shall be set > + * based on trace length differences from their > + * layout. > + * Mismatches up to 25% or tCK (clock period) are > + * allowed, so the value in the filed doesn’t > have > + * to be very accurate. > + * > + * - 0x2 (b'10) - RDLVL_DL_0/1 - refers to adjusting the DQS strobe in > relation > + * to the DQ signals so that the strobe edge is > + * centered in the window of valid read data. > + * > + * - 0x3 (b'11) - RDLVL_GTDL_0/1 - refers to the delay the PHY uses to > un-gate > + * the Read DQS strobe pad from the time that the > + * PHY enables the pad to input the strobe > signal. > + * > + */ > +static int ddr_cal_get_first_edge_index(unsigned long *bmap, enum edge e, > + int samples, int start, int max) > +{ > + int i, ret = -1; > + > + /* > + * We look only for the first value (and filter out > + * some wrong data) > + */ > + switch (e) { > + case RISING_EDGE: > + for (i = start; i <= max - samples; i++) { > + if (test_bit(i, bmap)) { > + if (!test_bit(i - 1, bmap) && > + test_bit(i + 1, bmap) && > + test_bit(i + 2, bmap) && > + test_bit(i + 3, bmap)) { > + return i; > + } > + } > + } > + break; > + case FALLING_EDGE: > + for (i = start; i <= max - samples; i++) { > + if (!test_bit(i, bmap)) { > + if (test_bit(i - 1, bmap) && > + test_bit(i - 2, bmap) && > + test_bit(i - 3, bmap)) { > + return i; > + } > + } > + } > + } > + > + return ret; > +} > + > +static void bitmap_print(unsigned long *bmap, int max) > +{ > + int i; > + > + debug("BITMAP [0x%p]:\n", bmap); > + for (i = 0; i <= max; i++) { > + debug("%d ", test_bit(i, bmap) ? 1 : 0); > + if (i && (i % 32) == (32 - 1)) > + debug("\n"); > + } > + debug("\n"); > +} > + > +#define sw_leveling_op_done \ > + while (!(readl(&ddrmr->cr[94]) & DDRMC_CR94_SWLVL_OP_DONE)) > + > +#define sw_leveling_load_value \ > + do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_LOAD, \ > + DDRMC_CR93_SWLVL_LOAD); } while (0) > + > +#define sw_leveling_start \ > + do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_START, \ > + DDRMC_CR93_SWLVL_START); } while (0) > + > +#define sw_leveling_exit \ > + do { clrsetbits_le32(&ddrmr->cr[94], DDRMC_CR93_SWLVL_EXIT, \ > + DDRMC_CR93_SWLVL_EXIT); } while (0) > + > +/* > + * RDLVL_DL calibration: > + * > + * NXP is _NOT_ recommending performing the leveling at each > + * boot. Instead - one shall run this procedure on new boards > + * and then use hardcoded values. > + * > + */ > +static int ddrmc_cal_dqs_to_dq(struct ddrmr_regs *ddrmr) > +{ > + DECLARE_BITMAP(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); > + int rdlvl_dl_0_min = -1, rdlvl_dl_0_max = -1; > + int rdlvl_dl_1_min = -1, rdlvl_dl_1_max = -1; > + int rdlvl_dl_0, rdlvl_dl_1; > + u8 swlvl_rsp; > + u32 tmp; > + int i; > + > + /* Read defaults */ > + u16 rdlvl_dl_0_def = > + (readl(&ddrmr->cr[105]) >> RDLVL_DL_O_OFF) & 0xFFFF; > + u16 rdlvl_dl_1_def = readl(&ddrmr->cr[110]) & 0xFFFF; > + > + debug("\nRDLVL: ======================\n"); > + debug("RDLVL: DQS to DQ (RDLVL)\n"); > + > + debug("RDLVL: RDLVL_DL_0_DFL:\t 0x%x\n", rdlvl_dl_0_def); > + debug("RDLVL: RDLVL_DL_1_DFL:\t 0x%x\n", rdlvl_dl_1_def); > + > + /* > + * Set/Read setup for calibration > + * > + * Values necessary for leveling from Vybrid RM [2] - page 1600 > + */ > + writel(0x40703030, &ddrmr->cr[144]); > + writel(0x40, &ddrmr->cr[145]); > + writel(0x40, &ddrmr->cr[146]); > + > + tmp = readl(&ddrmr->cr[144]); > + debug("RDLVL: PHY_RDLVL_RES:\t 0x%x\n", (tmp >> 24) & 0xFF);// set 0x40 > + debug("RDLVL: PHY_RDLV_LOAD:\t 0x%x\n", (tmp >> 16) & 0xFF);// set 0x70 > + debug("RDLVL: PHY_RDLV_DLL:\t 0x%x\n", (tmp >> 8) & 0xFF); // set 0x30 > + debug("RDLVL: PHY_RDLV_EN:\t 0x%x\n", tmp & 0xFF); //set 0x30 > + > + tmp = readl(&ddrmr->cr[145]); > + debug("RDLVL: PHY_RDLV_RR:\t 0x%x\n", tmp & 0x3FF); //set 0x40 > + > + tmp = readl(&ddrmr->cr[146]); > + debug("RDLVL: PHY_RDLV_RESP:\t 0x%x\n", tmp); //set 0x40 > + > + /* > + * Program/read the leveling edge RDLVL_EDGE = 0 > + * > + * 0x00 is the correct output on SWLVL_RSP_X > + * If by any chance 1s are visible -> wrong number read > + */ > + clrbits_le32(&ddrmr->cr[101], DDRMC_CR101_PHY_RDLVL_EDGE); > + > + tmp = readl(&ddrmr->cr[101]); > + debug("RDLVL: PHY_RDLVL_EDGE:\t 0x%x\n", > + (tmp >> PHY_RDLVL_EDGE) & 0x1); //set 0 > + > + /* Program Leveling mode - CR93[SW_LVL_MODE] to ’b10 */ > + clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SW_LVL_MODE(0x3), > + DDRMC_CR93_SW_LVL_MODE(0x2)); > + tmp = readl(&ddrmr->cr[93]); > + debug("RDLVL: SW_LVL_MODE:\t 0x%x\n", (tmp >> SW_LVL_MODE) & 0x3); > + > + /* Start procedure - CR93[SWLVL_START] to ’b1 */ > + sw_leveling_start; > + > + /* Poll CR94[SWLVL_OP_DONE] */ > + sw_leveling_op_done; > + > + /* > + * Program delays for RDLVL_DL_0 > + * > + * The procedure is to increase the delay values from 0 to 0xFF > + * and read the response from the DDRMC > + */ > + debug("\nRDLVL: ---> RDLVL_DL_0\n"); > + bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); > + > + for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) { > + clrsetbits_le32(&ddrmr->cr[105], 0xFFFF << RDLVL_DL_O_OFF, > + i << RDLVL_DL_O_OFF); > + > + /* Load values CR93[SWLVL_LOAD] to ’b1 */ > + sw_leveling_load_value; > + > + /* Poll CR94[SWLVL_OP_DONE] */ > + sw_leveling_op_done; > + > + /* > + * Read Responses - SWLVL_RESP_0 > + * > + * The 0x00 (correct response when PHY_RDLVL_EDGE = 0) > + * -> 1 in the bit vector > + */ > + swlvl_rsp = (readl(&ddrmr->cr[94]) >> SWLVL_RESP_0) & 0xF; > + if (swlvl_rsp == 0) > + generic_set_bit(i, rdlvl_rsp); > + } > + > + bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY); > + > + /* > + * First test for rising edge 0x0 -> 0x1 in bitmap > + */ > + rdlvl_dl_0_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE, > + N_SAMPLES, N_SAMPLES, > + DDRMC_DQS_DQ_MAX_DELAY); > + > + /* > + * Secondly test for falling edge 0x1 -> 0x0 in bitmap > + */ > + rdlvl_dl_0_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE, > + N_SAMPLES, rdlvl_dl_0_min, > + DDRMC_DQS_DQ_MAX_DELAY); > + > + debug("RDLVL: DL_0 min: %d [0x%x] DL_0 max: %d [0x%x]\n", > + rdlvl_dl_0_min, rdlvl_dl_0_min, rdlvl_dl_0_max, rdlvl_dl_0_max); > + rdlvl_dl_0 = (rdlvl_dl_0_max - rdlvl_dl_0_min) / 2; > + > + if (rdlvl_dl_0_max == -1 || rdlvl_dl_0_min == -1 || rdlvl_dl_0 <= 0) { > + debug("RDLVL: The DQS to DQ delay cannot be found!\n"); > + debug("RDLVL: Using default - slice 0: %d!\n", rdlvl_dl_0_def); > + rdlvl_dl_0 = rdlvl_dl_0_def; > + } > + > + debug("\nRDLVL: ---> RDLVL_DL_1\n"); > + bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); > + > + for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) { > + clrsetbits_le32(&ddrmr->cr[110], 0xFFFF << RDLVL_DL_1_OFF, > + i << RDLVL_DL_1_OFF); > + > + /* Load values CR93[SWLVL_LOAD] to ’b1 */ > + sw_leveling_load_value; > + > + /* Poll CR94[SWLVL_OP_DONE] */ > + sw_leveling_op_done; > + > + /* > + * Read Responses - SWLVL_RESP_1 > + * > + * The 0x00 (correct response when PHY_RDLVL_EDGE = 0) > + * -> 1 in the bit vector > + */ > + swlvl_rsp = (readl(&ddrmr->cr[95]) >> SWLVL_RESP_1) & 0xF; > + if (swlvl_rsp == 0) > + generic_set_bit(i, rdlvl_rsp); > + } > + > + bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY); > + > + /* > + * First test for rising edge 0x0 -> 0x1 in bitmap > + */ > + rdlvl_dl_1_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE, > + N_SAMPLES, N_SAMPLES, > + DDRMC_DQS_DQ_MAX_DELAY); > + > + /* > + * Secondly test for falling edge 0x1 -> 0x0 in bitmap > + */ > + rdlvl_dl_1_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE, > + N_SAMPLES, rdlvl_dl_1_min, > + DDRMC_DQS_DQ_MAX_DELAY); > + > + debug("RDLVL: DL_1 min: %d [0x%x] DL_1 max: %d [0x%x]\n", > + rdlvl_dl_1_min, rdlvl_dl_1_min, rdlvl_dl_1_max, rdlvl_dl_1_max); > + rdlvl_dl_1 = (rdlvl_dl_1_max - rdlvl_dl_1_min) / 2; > + > + if (rdlvl_dl_1_max == -1 || rdlvl_dl_1_min == -1 || rdlvl_dl_1 <= 0) { > + debug("RDLVL: The DQS to DQ delay cannot be found!\n"); > + debug("RDLVL: Using default - slice 1: %d!\n", rdlvl_dl_1_def); > + rdlvl_dl_1 = rdlvl_dl_1_def; > + } > + > + debug("RDLVL: CALIBRATED: rdlvl_dl_0: 0x%x\t rdlvl_dl_1: 0x%x\n", > + rdlvl_dl_0, rdlvl_dl_1); > + > + /* Write new delay values */ > + writel((rdlvl_dl_0 << RDLVL_DL_O_OFF), &ddrmr->cr[105]); > + writel((rdlvl_dl_1 << RDLVL_DL_1_OFF), &ddrmr->cr[110]); > + > + sw_leveling_load_value; > + sw_leveling_op_done; > + > + /* Exit procedure - CR94[SWLVL_EXIT] to ’b1 */ > + sw_leveling_exit; > + > + /* Poll CR94[SWLVL_OP_DONE] */ > + sw_leveling_op_done; > + > + return 0; > +} > + > +/* > + * WRLVL_DL calibration: > + * > + * For non-flyback memory architecture - where one have a single DDR3 x16 > + * memory - it is NOT necessary to perform "Write Leveling" > + * [3] 'Vybrid DDR3 write leveling' https://community.nxp.com/thread/429362 > + * > + */ > + > +int ddrmc_calibration(struct ddrmr_regs *ddrmr) > +{ > + ddrmc_cal_dqs_to_dq(ddrmr); > + > + return 0; > +} > diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.h > b/arch/arm/mach-imx/ddrmc-vf610-calibration.h > new file mode 100644 > index 0000000000..7bc53236be > --- /dev/null > +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.h > @@ -0,0 +1,59 @@ > +/* SPDX-License-Identifier: GPL-2.0+ */ > +/* > + * ddrmc DDR3 calibration code for NXP's VF610 > + * > + * Copyright (C) 2018 DENX Software Engineering > + * Lukasz Majewski, DENX Software Engineering, lu...@denx.de > + * > + */ > + > +#ifndef __DDRMC_VF610_CALIBRATOIN_H_ > +#define __DDRMC_VF610_CALIBRATOIN_H_ > + > +/* > + * Number of "samples" in the calibration bitmap > + * to be considered during calibration. > + */ > +#define N_SAMPLES 3 > + > +/* > + * Constants to indicate if we are looking for a rising or > + * falling edge in the calibration bitmap > + */ > +enum edge { > + FALLING_EDGE = 1, > + RISING_EDGE > +}; > + > +/* > + * The max number of delay elements when DQS to DQ setting > + */ > +#define DDRMC_DQS_DQ_MAX_DELAY 0xFF > + > +/* Bits offsets for DDRMC_CR registers */ > +/* CR101 */ > +#define PHY_RDLVL_EDGE 24 > +/* CR93 */ > +#define SW_LVL_MODE 8 > +/* CR94 */ > +#define SWLVL_RESP_0 24 > +/* CR95 */ > +#define SWLVL_RESP_1 0 > +/* CR105 */ > +#define RDLVL_DL_O_OFF 8 > +/* CR110 */ > +#define RDLVL_DL_1_OFF 0 Should we not move those defines to arch/arm/include/asm/arch-vf610/imx-regs.h too? -- Stefan > + > +/** > + * ddrmc_calibration - Vybrid's (VF610) DDR3 calibration code > + * > + * This function is calculating proper memory controller values > + * during run time. > + * > + * @param ddrmr_regs - memory controller registers > + * > + * @return 0 on success, otherwise error code > + */ > +int ddrmc_calibration(struct ddrmr_regs *ddrmr); > + > +#endif /* __DDRMC_VF610_CALIBRATOIN_H_ */ _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
