Author: mmel Date: Sat Sep 19 11:06:41 2020 New Revision: 365899 URL: https://svnweb.freebsd.org/changeset/base/365899
Log: Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions. One problem with the bus_space_read_N() and bus_space_write_N() family of functions is that they provide no protection against exceptions which can occur when no physical hardware or device responds to the read or write cycles. In such a situation, the system typically would panic due to a kernel-mode bus error. The bus_space_peek_N() and bus_space_poke_N() family of functions provide a mechanism to handle these exceptions gracefully without the risk of crashing the system. Typical example is access to PCI(e) configuration space in bus enumeration function on badly implemented PCI(e) root complexes (RK3399 or Neoverse N1 N1SDP and/or access to PCI(e) register when device is in deep sleep state. This commit adds a real implementation for arm64 only. The remaining architectures have bus_space_peek()/bus_space_poke() emulated by using bus_space_read()/bus_space_write() (without exception handling). MFC after: 1 month Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D25371 Modified: head/share/man/man9/bus_space.9 head/sys/arm/include/bus.h head/sys/arm64/arm64/bus_machdep.c head/sys/arm64/arm64/bus_space_asm.S head/sys/arm64/arm64/trap.c head/sys/arm64/include/bus.h head/sys/arm64/include/md_var.h head/sys/kern/subr_csan.c head/sys/mips/include/bus.h head/sys/powerpc/include/bus.h head/sys/riscv/include/bus.h head/sys/sys/_cscan_bus.h head/sys/sys/bus.h head/sys/tools/bus_macro.sh head/sys/x86/include/bus.h Modified: head/share/man/man9/bus_space.9 ============================================================================== --- head/share/man/man9/bus_space.9 Sat Sep 19 02:15:56 2020 (r365898) +++ head/share/man/man9/bus_space.9 Sat Sep 19 11:06:41 2020 (r365899) @@ -52,7 +52,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 15, 2017 +.Dd July 7, 2020 .Dt BUS_SPACE 9 .Os .Sh NAME @@ -68,6 +68,14 @@ .Nm bus_space_copy_region_stream_8 , .Nm bus_space_free , .Nm bus_space_map , +.Nm bus_space_peek_1 , +.Nm bus_space_peek_2 , +.Nm bus_space_peek_4 , +.Nm bus_space_peek_8 , +.Nm bus_space_poke_1 , +.Nm bus_space_poke_2 , +.Nm bus_space_poke_4 , +.Nm bus_space_poke_8 , .Nm bus_space_read_1 , .Nm bus_space_read_2 , .Nm bus_space_read_4 , @@ -161,6 +169,46 @@ .Fo bus_space_free .Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t size" .Fc +.Ft int +.Fo bus_space_peek_1 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_peek_2 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_peek_4 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_peek_8 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_poke_1 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_poke_2 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_poke_4 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc +.Ft int +.Fo bus_space_poke_8 +.Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" +.Fa "uint8_t *datap" +.Fc .Ft uint8_t .Fo bus_space_read_1 .Fa "bus_space_tag_t space" "bus_space_handle_t handle" "bus_size_t offset" @@ -1124,6 +1172,105 @@ If they would fail (e.g.\& because of an argument error), that indicates a software bug which should cause a panic. In that case, they will never return. +.Sh PROBING BUS SPACE FOR HARDWARE WHICH MAY NOT RESPOND +One problem with the +.Fn bus_space_read_N +and +.Fn bus_space_write_N +family of functions is that they provide no protection against +exceptions which can occur when no physical hardware or +device responds to the read or write cycles. +In such a situation, the system typically would panic due to a kernel-mode +bus error. +The +.Fn bus_space_peek_N +and +.Fn bus_space_poke_N +family of functions provide a mechanism to handle these exceptions +gracefully without the risk of crashing the system. +.Pp +As with +.Fn bus_space_read_N +and +.Fn bus_space_write_N , +the peek and poke functions provide the ability to read and +write 1, 2, 4, and 8 byte data items on busses which support those +access sizes. +All of the constraints specified in the descriptions of the +.Fn bus_space_read_N +and +.Fn bus_space_write_N +functions also apply to +.Fn bus_space_peek_N +and +.Fn bus_space_poke_N . +.Pp +In addition, explicit calls to the +.Fn bus_space_barrier +function are not required as the implementation will ensure all +pending operations complete before the peek or poke operation starts. +The implementation will also ensure that the peek or poke operations +complete before returning. +.Pp +The return value indicates the outcome of the peek or poke operation. +A return value of zero implies that a hardware device is +responding to the operation at the specified offset in the bus space. +A non-zero return value indicates that the kernel intercepted a +hardware exception (e.g., bus error) when the peek or poke operation +was attempted. +Note that some busses are incapable of generating exceptions when +non-existent hardware is accessed. +In such cases, these functions will always return zero and the value of +the data read by +.Fn bus_space_peek_N +will be unspecified. +.Pp +Finally, it should be noted that at this time the +.Fn bus_space_peek_N +and +.Fn bus_space_poke_N +functions are not re-entrant and should not, therefore, be used +from within an interrupt service routine. +This constraint may be removed at some point in the future. +.Pp +.Bl -ohang -compact +.It Fn bus_space_peek_1 "space" "handle" "offset" "datap" +.It Fn bus_space_peek_2 "space" "handle" "offset" "datap" +.It Fn bus_space_peek_4 "space" "handle" "offset" "datap" +.It Fn bus_space_peek_8 "space" "handle" "offset" "datap" +.Pp +The +.Fn bus_space_peek_N +family of functions cautiously read a 1, 2, 4, or 8 byte data item from +the offset specified by +.Fa offset +in the region specified by +.Fa handle +of the bus space specified by +.Fa space . +The data item read is stored in the location pointed to by +.Fa datap . +It is permissible for +.Fa datap +to be NULL, in which case the data item will be discarded after being read. +.Pp +.It Fn bus_space_poke_1 "space" "handle" "offset" "value" +.It Fn bus_space_poke_2 "space" "handle" "offset" "value" +.It Fn bus_space_poke_4 "space" "handle" "offset" "value" +.It Fn bus_space_poke_8 "space" "handle" "offset" "value" +.Pp +The +.Fn bus_space_poke_N +family of functions cautiously write a 1, 2, 4, or 8 byte data item +specified by +.Fa value +to the offset specified by +.Fa offset +in the region specified by +.Fa handle +of the bus space specified by +.Fa space . +.El .Sh BARRIERS In order to allow high-performance buffering implementations to avoid bus activity on every operation, read and write ordering should be specified Modified: head/sys/arm/include/bus.h ============================================================================== --- head/sys/arm/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -747,6 +747,33 @@ void generic_bs_unimplemented(void); #define BUS_SPACE_UNRESTRICTED (~0) +#define BUS_PEEK_FUNC(width, type) \ + static inline int \ + bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + type tmp; \ + tmp = bus_space_read_##width(tag, hnd, offset); \ + return (0); \ + } +BUS_PEEK_FUNC(1, uint8_t) +BUS_PEEK_FUNC(2, uint16_t) +BUS_PEEK_FUNC(4, uint32_t) +BUS_PEEK_FUNC(8, uint64_t) + +#define BUS_POKE_FUNC(width, type) \ + static inline int \ + bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write_##width(tag, hnd, offset, value); \ + return (0); \ + } +BUS_POKE_FUNC(1, uint8_t) +BUS_POKE_FUNC(2, uint16_t) +BUS_POKE_FUNC(4, uint32_t) +BUS_POKE_FUNC(8, uint64_t) + #include <machine/bus_dma.h> /* Modified: head/sys/arm64/arm64/bus_machdep.c ============================================================================== --- head/sys/arm64/arm64/bus_machdep.c Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm64/arm64/bus_machdep.c Sat Sep 19 11:06:41 2020 (r365899) @@ -83,6 +83,16 @@ void generic_bs_wr_4(void *, bus_space_handle_t, bus_s void generic_bs_wr_8(void *, bus_space_handle_t, bus_size_t, const uint64_t *, bus_size_t); +int generic_bs_peek_1(void *, bus_space_handle_t, bus_size_t , uint8_t *); +int generic_bs_peek_2(void *, bus_space_handle_t, bus_size_t , uint16_t *); +int generic_bs_peek_4(void *, bus_space_handle_t, bus_size_t , uint32_t *); +int generic_bs_peek_8(void *, bus_space_handle_t, bus_size_t , uint64_t *); + +int generic_bs_poke_1(void *, bus_space_handle_t, bus_size_t, uint8_t); +int generic_bs_poke_2(void *, bus_space_handle_t, bus_size_t, uint16_t); +int generic_bs_poke_4(void *, bus_space_handle_t, bus_size_t, uint32_t); +int generic_bs_poke_8(void *, bus_space_handle_t, bus_size_t, uint64_t); + static int generic_bs_map(void *t, bus_addr_t bpa, bus_size_t size, int flags, bus_space_handle_t *bshp) @@ -223,6 +233,18 @@ struct bus_space memmap_bus = { .bs_wr_2_s = NULL, .bs_wr_4_s = NULL, .bs_wr_8_s = NULL, + + /* peek */ + .bs_peek_1 = generic_bs_peek_1, + .bs_peek_2 = generic_bs_peek_2, + .bs_peek_4 = generic_bs_peek_4, + .bs_peek_8 = generic_bs_peek_8, + + /* poke */ + .bs_poke_1 = generic_bs_poke_1, + .bs_poke_2 = generic_bs_poke_2, + .bs_poke_4 = generic_bs_poke_4, + .bs_poke_8 = generic_bs_poke_8, }; #ifdef FDT Modified: head/sys/arm64/arm64/bus_space_asm.S ============================================================================== --- head/sys/arm64/arm64/bus_space_asm.S Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm64/arm64/bus_space_asm.S Sat Sep 19 11:06:41 2020 (r365899) @@ -26,7 +26,6 @@ */ #include <machine/asm.h> - __FBSDID("$FreeBSD$"); ENTRY(generic_bs_r_1) @@ -397,3 +396,84 @@ ENTRY(generic_bs_wr_8) 2: ret END(generic_bs_wr_8) + +ENTRY(generic_bs_fault) + mov x0, #-1 + ret +END(bus_fault) + +ENTRY(generic_bs_peek_1) + .globl generic_bs_peek_1f +generic_bs_peek_1f: + ldrb w0, [x1, x2] /* Checked instruction */ + dsb sy + strb w0,[x3] + mov x0, #0 + ret +END(generic_bs_peek_1) + +ENTRY(generic_bs_peek_2) + .globl generic_bs_peek_2f +generic_bs_peek_2f: + ldrh w0, [x1, x2] /* Checked instruction */ + dsb sy + strh w0,[x3] + mov x0, #0 + ret +END(generic_bs_peek_2) + +ENTRY(generic_bs_peek_4) + .globl generic_bs_peek_4f +generic_bs_peek_4f: + ldr w0, [x1, x2] /* Checked instruction */ + dsb sy + str w0,[x3] + mov x0, #0 + ret +END(generic_bs_peek_4) + +ENTRY(generic_bs_peek_8) + .globl generic_bs_peek_8f +generic_bs_peek_8f: + ldr x0, [x1, x2] /* Checked instruction */ + dsb sy + str x0,[x3] + mov x0, #0 + ret +END(generic_bs_peek_8) + +ENTRY(generic_bs_poke_1) + .globl generic_bs_poke_1f +generic_bs_poke_1f: + strb w3, [x1, x2] /* Checked instruction */ + dsb sy + mov x0, #0 + ret +END(generic_bs_poke_1) + +ENTRY(generic_bs_poke_2) + .globl generic_bs_poke_2f +generic_bs_poke_2f: + strh w3, [x1, x2] /* Checked instruction */ + dsb sy + mov x0, #0 + ret +END(generic_bs_poke_2) + +ENTRY(generic_bs_poke_4) + .globl generic_bs_poke_4f +generic_bs_poke_4f: + str w3, [x1, x2] /* Checked instruction */ + dsb sy + mov x0, #0 + ret +END(generic_bs_poke_4) + +ENTRY(generic_bs_poke_8) + .globl generic_bs_poke_8f +generic_bs_poke_8f: + str x3, [x1, x2] /* Checked instruction */ + dsb sy + mov x0, #0 + ret +END(generic_bs_poke_8) Modified: head/sys/arm64/arm64/trap.c ============================================================================== --- head/sys/arm64/arm64/trap.c Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm64/arm64/trap.c Sat Sep 19 11:06:41 2020 (r365899) @@ -50,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include <vm/vm_extern.h> #include <machine/frame.h> +#include <machine/md_var.h> #include <machine/pcb.h> #include <machine/pcpu.h> #include <machine/undefined.h> @@ -88,6 +89,7 @@ typedef void (abort_handler)(struct thread *, struct t static abort_handler align_abort; static abort_handler data_abort; +static abort_handler external_abort; static abort_handler *abort_handlers[] = { [ISS_DATA_DFSC_TF_L0] = data_abort, @@ -101,6 +103,7 @@ static abort_handler *abort_handlers[] = { [ISS_DATA_DFSC_PF_L2] = data_abort, [ISS_DATA_DFSC_PF_L3] = data_abort, [ISS_DATA_DFSC_ALIGN] = align_abort, + [ISS_DATA_DFSC_EXT] = external_abort, }; static __inline void @@ -154,6 +157,28 @@ cpu_fetch_syscall_args(struct thread *td) #include "../../kern/subr_syscall.c" +/* + * Test for fault generated by given access instruction in + * bus_peek_<foo> or bus_poke_<foo> bus function. + */ +extern uint32_t generic_bs_peek_1f, generic_bs_peek_2f; +extern uint32_t generic_bs_peek_4f, generic_bs_peek_8f; +extern uint32_t generic_bs_poke_1f, generic_bs_poke_2f; +extern uint32_t generic_bs_poke_4f, generic_bs_poke_8f; + +static bool +test_bs_fault(void *addr) +{ + return (addr == &generic_bs_peek_1f || + addr == &generic_bs_peek_2f || + addr == &generic_bs_peek_4f || + addr == &generic_bs_peek_8f || + addr == &generic_bs_poke_1f || + addr == &generic_bs_poke_2f || + addr == &generic_bs_poke_4f || + addr == &generic_bs_poke_8f); +} + static void svc_handler(struct thread *td, struct trapframe *frame) { @@ -182,6 +207,26 @@ align_abort(struct thread *td, struct trapframe *frame call_trapsignal(td, SIGBUS, BUS_ADRALN, (void *)frame->tf_elr, ESR_ELx_EXCEPTION(frame->tf_esr)); userret(td, frame); +} + + +static void +external_abort(struct thread *td, struct trapframe *frame, uint64_t esr, + uint64_t far, int lower) +{ + + /* + * Try to handle synchronous external aborts caused by + * bus_space_peek() and/or bus_space_poke() functions. + */ + if (!lower && test_bs_fault((void *)frame->tf_elr)) { + frame->tf_elr = (uint64_t)generic_bs_fault; + return; + } + + print_registers(frame); + printf(" far: %16lx\n", far); + panic("Unhandled EL%d external data abort", lower ? 0: 1); } static void Modified: head/sys/arm64/include/bus.h ============================================================================== --- head/sys/arm64/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm64/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -259,6 +259,24 @@ struct bus_space { bus_size_t, const u_int32_t *, bus_size_t); void (*bs_wr_8_s) (void *, bus_space_handle_t, bus_size_t, const u_int64_t *, bus_size_t); + /* peek */ + int (*bs_peek_1)(void *, bus_space_handle_t, + bus_size_t , uint8_t *); + int (*bs_peek_2)(void *, bus_space_handle_t, + bus_size_t , uint16_t *); + int (*bs_peek_4)(void *, bus_space_handle_t, + bus_size_t , uint32_t *); + int (*bs_peek_8)(void *, bus_space_handle_t, + bus_size_t , uint64_t *); + /* poke */ + int (*bs_poke_1)(void *, bus_space_handle_t, + bus_size_t, uint8_t); + int (*bs_poke_2)(void *, bus_space_handle_t, + bus_size_t, uint16_t); + int (*bs_poke_4)(void *, bus_space_handle_t, + bus_size_t, uint32_t); + int (*bs_poke_8)(void *, bus_space_handle_t, + bus_size_t, uint64_t); }; /* @@ -283,6 +301,10 @@ struct bus_space { (*(t)->__bs_opname_s(r,sz))((t)->bs_cookie, h, o) #define __bs_ws_s(sz, t, h, o, v) \ (*(t)->__bs_opname_s(w,sz))((t)->bs_cookie, h, o, v) +#define __bs_peek(sz, t, h, o, vp) \ + (*(t)->__bs_opname(peek, sz))((t)->bs_cookie, h, o, vp) +#define __bs_poke(sz, t, h, o, v) \ + (*(t)->__bs_opname(poke, sz))((t)->bs_cookie, h, o, v) #define __bs_nonsingle_s(type, sz, t, h, o, a, c) \ (*(t)->__bs_opname_s(type,sz))((t)->bs_cookie, h, o, a, c) @@ -456,6 +478,22 @@ struct bus_space { __bs_copy(4, t, h1, o1, h2, o2, c) #define bus_space_copy_region_8(t, h1, o1, h2, o2, c) \ __bs_copy(8, t, h1, o1, h2, o2, c) + +/* + * Poke (checked write) operations. + */ +#define bus_space_poke_1(t, h, o, v) __bs_poke(1, (t), (h), (o), (v)) +#define bus_space_poke_2(t, h, o, v) __bs_poke(2, (t), (h), (o), (v)) +#define bus_space_poke_4(t, h, o, v) __bs_poke(4, (t), (h), (o), (v)) +#define bus_space_poke_8(t, h, o, v) __bs_poke(8, (t), (h), (o), (v)) + +/* + * Peek (checked read) operations. + */ +#define bus_space_peek_1(t, h, o, vp) __bs_peek(1, (t), (h), (o), (vp)) +#define bus_space_peek_2(t, h, o, vp) __bs_peek(2, (t), (h), (o), (vp)) +#define bus_space_peek_4(t, h, o, vp) __bs_peek(4, (t), (h), (o), (vp)) +#define bus_space_peek_8(t, h, o, vp) __bs_peek(8, (t), (h), (o), (vp)) #endif Modified: head/sys/arm64/include/md_var.h ============================================================================== --- head/sys/arm64/include/md_var.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/arm64/include/md_var.h Sat Sep 19 11:06:41 2020 (r365899) @@ -48,5 +48,14 @@ void busdma_swi(void); void dump_add_page(vm_paddr_t); void dump_drop_page(vm_paddr_t); int minidumpsys(struct dumperinfo *); +void generic_bs_fault(void) __asm(__STRING(generic_bs_fault)); +void generic_bs_peek_1(void) __asm(__STRING(generic_bs_peek_1)); +void generic_bs_peek_2(void) __asm(__STRING(generic_bs_peek_2)); +void generic_bs_peek_4(void) __asm(__STRING(generic_bs_peek_4)); +void generic_bs_peek_8(void) __asm(__STRING(generic_bs_peek_8)); +void generic_bs_poke_1(void) __asm(__STRING(generic_bs_poke_1)); +void generic_bs_poke_2(void) __asm(__STRING(generic_bs_poke_2)); +void generic_bs_poke_4(void) __asm(__STRING(generic_bs_poke_4)); +void generic_bs_poke_8(void) __asm(__STRING(generic_bs_poke_8)); #endif /* !_MACHINE_MD_VAR_H_ */ Modified: head/sys/kern/subr_csan.c ============================================================================== --- head/sys/kern/subr_csan.c Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/kern/subr_csan.c Sat Sep 19 11:06:41 2020 (r365899) @@ -878,3 +878,33 @@ CSAN_BUS_SET_FUNC(multi_stream, 8, uint64_t) CSAN_BUS_SET_FUNC(region_stream, 8, uint64_t) #endif #endif + +#define CSAN_BUS_PEEK_FUNC(width, type) \ + int kcsan_bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + kcsan_access((uintptr_t)value, sizeof(type), true, false, \ + __RET_ADDR); \ + return (bus_space_peek_##width(tag, hnd, offset, value)); \ + } + +CSAN_BUS_PEEK_FUNC(1, uint8_t) +CSAN_BUS_PEEK_FUNC(2, uint16_t) +CSAN_BUS_PEEK_FUNC(4, uint32_t) +#if !defined(__i386__) +CSAN_BUS_PEEK_FUNC(8, uint64_t) +#endif + +#define CSAN_BUS_POKE_FUNC(width, type) \ + int kcsan_bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + return (bus_space_poke_##width(tag, hnd, offset, value)); \ + } + +CSAN_BUS_POKE_FUNC(1, uint8_t) +CSAN_BUS_POKE_FUNC(2, uint16_t) +CSAN_BUS_POKE_FUNC(4, uint32_t) +#if !defined(__i386__) +CSAN_BUS_POKE_FUNC(8, uint64_t) +#endif Modified: head/sys/mips/include/bus.h ============================================================================== --- head/sys/mips/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/mips/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -689,6 +689,33 @@ void __bs_c(f,_bs_c_8) (void *t, bus_space_handle_t bs bs_c_4_proto(f); \ bs_c_8_proto(f); +#define BUS_PEEK_FUNC(width, type) \ + static inline int \ + bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + type tmp; \ + tmp = bus_space_read_##width(tag, hnd, offset); \ + return (0); \ + } +BUS_PEEK_FUNC(1, uint8_t) +BUS_PEEK_FUNC(2, uint16_t) +BUS_PEEK_FUNC(4, uint32_t) +BUS_PEEK_FUNC(8, uint64_t) + +#define BUS_POKE_FUNC(width, type) \ + static inline int \ + bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write_##width(tag, hnd, offset, value); \ + return (0); \ + } +BUS_POKE_FUNC(1, uint8_t) +BUS_POKE_FUNC(2, uint16_t) +BUS_POKE_FUNC(4, uint32_t) +BUS_POKE_FUNC(8, uint64_t) + #define BUS_SPACE_ALIGNED_POINTER(p, t) ALIGNED_POINTER(p, t) #define BUS_SPACE_MAXADDR_24BIT 0xFFFFFF Modified: head/sys/powerpc/include/bus.h ============================================================================== --- head/sys/powerpc/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/powerpc/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -462,6 +462,33 @@ extern struct bus_space bs_le_tag; __bs_copy(s_8, t, h1, o1, h2, o2, c) #endif +#define BUS_PEEK_FUNC(width, type) \ + static inline int \ + bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + type tmp; \ + tmp = bus_space_read_##width(tag, hnd, offset); \ + return (0); \ + } +BUS_PEEK_FUNC(1, uint8_t) +BUS_PEEK_FUNC(2, uint16_t) +BUS_PEEK_FUNC(4, uint32_t) +BUS_PEEK_FUNC(8, uint64_t) + +#define BUS_POKE_FUNC(width, type) \ + static inline int \ + bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write_##width(tag, hnd, offset, value); \ + return (0); \ + } +BUS_POKE_FUNC(1, uint8_t) +BUS_POKE_FUNC(2, uint16_t) +BUS_POKE_FUNC(4, uint32_t) +BUS_POKE_FUNC(8, uint64_t) + #include <machine/bus_dma.h> #endif /* _MACHINE_BUS_H_ */ Modified: head/sys/riscv/include/bus.h ============================================================================== --- head/sys/riscv/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/riscv/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -451,6 +451,33 @@ struct bus_space { #define bus_space_copy_region_8(t, h1, o1, h2, o2, c) \ __bs_copy(8, t, h1, o1, h2, o2, c) +#define BUS_PEEK_FUNC(width, type) \ + static inline int \ + bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + type tmp; \ + tmp = bus_space_read_##width(tag, hnd, offset); \ + return (0); \ + } +BUS_PEEK_FUNC(1, uint8_t) +BUS_PEEK_FUNC(2, uint16_t) +BUS_PEEK_FUNC(4, uint32_t) +BUS_PEEK_FUNC(8, uint64_t) + +#define BUS_POKE_FUNC(width, type) \ + static inline int \ + bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write_##width(tag, hnd, offset, value); \ + return (0); \ + } +BUS_POKE_FUNC(1, uint8_t) +BUS_POKE_FUNC(2, uint16_t) +BUS_POKE_FUNC(4, uint32_t) +BUS_POKE_FUNC(8, uint64_t) + #include <machine/bus_dma.h> #endif /* _MACHINE_BUS_H_ */ Modified: head/sys/sys/_cscan_bus.h ============================================================================== --- head/sys/sys/_cscan_bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/sys/_cscan_bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -77,11 +77,21 @@ bus_space_handle_t, bus_size_t, bus_space_handle_t, \ bus_size_t, bus_size_t); +#define KCSAN_BS_PEEK(width, type) \ + int kcsan_bus_space_peek_##width(bus_space_tag_t, \ + bus_space_handle_t, bus_size_t, type *); + +#define KCSAN_BS_POKE(width, type) \ + int kcsan_bus_space_poke_##width(bus_space_tag_t, \ + bus_space_handle_t, bus_size_t, type); + #define KCSAN_BS(width, type) \ KCSAN_BS_READ(width, type); \ KCSAN_BS_WRITE(width, type); \ KCSAN_BS_SET(width, type); \ - KCSAN_BS_COPY(width, type) + KCSAN_BS_COPY(width, type) \ + KCSAN_BS_PEEK(width, type); \ + KCSAN_BS_POKE(width, type); KCSAN_BS(1, uint8_t); KCSAN_BS(2, uint16_t); @@ -127,6 +137,8 @@ void kcsan_bus_space_barrier(bus_space_tag_t, bus_spac #define bus_space_set_region_stream_1 kcsan_bus_space_set_region_stream_1 #define bus_space_copy_multi_1 kcsan_bus_space_copy_multi_1 #define bus_space_copy_multi_stream_1 kcsan_bus_space_copy_multi_stream_1 +#define bus_space_poke_1 kcsan_bus_space_poke_1 +#define bus_space_peek_1 kcsan_bus_space_peek_1 #define bus_space_read_2 kcsan_bus_space_read_2 #define bus_space_read_stream_2 kcsan_bus_space_read_stream_2 @@ -146,6 +158,8 @@ void kcsan_bus_space_barrier(bus_space_tag_t, bus_spac #define bus_space_set_region_stream_2 kcsan_bus_space_set_region_stream_2 #define bus_space_copy_multi_2 kcsan_bus_space_copy_multi_2 #define bus_space_copy_multi_stream_2 kcsan_bus_space_copy_multi_stream_2 +#define bus_space_poke_2 kcsan_bus_space_poke_2 +#define bus_space_peek_2 kcsan_bus_space_peek_2 #define bus_space_read_4 kcsan_bus_space_read_4 #define bus_space_read_stream_4 kcsan_bus_space_read_stream_4 @@ -165,6 +179,8 @@ void kcsan_bus_space_barrier(bus_space_tag_t, bus_spac #define bus_space_set_region_stream_4 kcsan_bus_space_set_region_stream_4 #define bus_space_copy_multi_4 kcsan_bus_space_copy_multi_4 #define bus_space_copy_multi_stream_4 kcsan_bus_space_copy_multi_stream_4 +#define bus_space_poke_4 kcsan_bus_space_poke_4 +#define bus_space_peek_4 kcsan_bus_space_peek_4 #define bus_space_read_8 kcsan_bus_space_read_8 #define bus_space_read_stream_8 kcsan_bus_space_read_stream_8 @@ -184,6 +200,9 @@ void kcsan_bus_space_barrier(bus_space_tag_t, bus_spac #define bus_space_set_region_stream_8 kcsan_bus_space_set_region_stream_8 #define bus_space_copy_multi_8 kcsan_bus_space_copy_multi_8 #define bus_space_copy_multi_stream_8 kcsan_bus_space_copy_multi_stream_8 +#define bus_space_poke_8 kcsan_bus_space_poke_8 +#define bus_space_peek_8 kcsan_bus_space_peek_8 + #endif /* !KCSAN_RUNTIME */ Modified: head/sys/sys/bus.h ============================================================================== --- head/sys/sys/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/sys/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -827,6 +827,10 @@ static __inline void varp ## _set_ ## var(device_t dev #define bus_barrier(r, o, l, f) \ bus_space_barrier((r)->r_bustag, (r)->r_bushandle, (o), (l), (f)) +#define bus_poke_1(r, o, v) \ + bus_space_poke_1((r)->r_bustag, (r)->r_bushandle, (o), (v)) +#define bus_peek_1(r, o, vp) \ + bus_space_peek_1((r)->r_bustag, (r)->r_bushandle, (o), (vp)) #define bus_read_1(r, o) \ bus_space_read_1((r)->r_bustag, (r)->r_bushandle, (o)) #define bus_read_multi_1(r, o, d, c) \ @@ -859,6 +863,10 @@ static __inline void varp ## _set_ ## var(device_t dev bus_space_write_multi_stream_1((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) #define bus_write_region_stream_1(r, o, d, c) \ bus_space_write_region_stream_1((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) +#define bus_poke_2(r, o, v) \ + bus_space_poke_2((r)->r_bustag, (r)->r_bushandle, (o), (v)) +#define bus_peek_2(r, o, vp) \ + bus_space_peek_2((r)->r_bustag, (r)->r_bushandle, (o), (vp)) #define bus_read_2(r, o) \ bus_space_read_2((r)->r_bustag, (r)->r_bushandle, (o)) #define bus_read_multi_2(r, o, d, c) \ @@ -891,6 +899,10 @@ static __inline void varp ## _set_ ## var(device_t dev bus_space_write_multi_stream_2((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) #define bus_write_region_stream_2(r, o, d, c) \ bus_space_write_region_stream_2((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) +#define bus_poke_4(r, o, v) \ + bus_space_poke_4((r)->r_bustag, (r)->r_bushandle, (o), (v)) +#define bus_peek_4(r, o, vp) \ + bus_space_peek_4((r)->r_bustag, (r)->r_bushandle, (o), (vp)) #define bus_read_4(r, o) \ bus_space_read_4((r)->r_bustag, (r)->r_bushandle, (o)) #define bus_read_multi_4(r, o, d, c) \ @@ -923,6 +935,10 @@ static __inline void varp ## _set_ ## var(device_t dev bus_space_write_multi_stream_4((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) #define bus_write_region_stream_4(r, o, d, c) \ bus_space_write_region_stream_4((r)->r_bustag, (r)->r_bushandle, (o), (d), (c)) +#define bus_poke_8(r, o, v) \ + bus_space_poke_8((r)->r_bustag, (r)->r_bushandle, (o), (v)) +#define bus_peek_8(r, o, vp) \ + bus_space_peek_8((r)->r_bustag, (r)->r_bushandle, (o), (vp)) #define bus_read_8(r, o) \ bus_space_read_8((r)->r_bustag, (r)->r_bushandle, (o)) #define bus_read_multi_8(r, o, d, c) \ Modified: head/sys/tools/bus_macro.sh ============================================================================== --- head/sys/tools/bus_macro.sh Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/tools/bus_macro.sh Sat Sep 19 11:06:41 2020 (r365899) @@ -55,7 +55,8 @@ for w in 1 2 4 8 do # macro copy_region_$w so dh do c # macro copy_region_stream_$w ? - # macro peek_$w + macro poke_$w o v + macro peek_$w o vp for s in "" stream_ do macro read_$s$w o Modified: head/sys/x86/include/bus.h ============================================================================== --- head/sys/x86/include/bus.h Sat Sep 19 02:15:56 2020 (r365898) +++ head/sys/x86/include/bus.h Sat Sep 19 11:06:41 2020 (r365899) @@ -1089,6 +1089,31 @@ bus_space_barrier(bus_space_tag_t tag __unused, bus_sp #define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) +#define BUS_PEEK_FUNC(width, type) \ + static inline int \ + bus_space_peek_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type *value) \ + { \ + type tmp; \ + tmp = bus_space_read_##width(tag, hnd, offset); \ + return (0); \ + } +BUS_PEEK_FUNC(1, uint8_t) +BUS_PEEK_FUNC(2, uint16_t) +BUS_PEEK_FUNC(4, uint32_t) + +#define BUS_POKE_FUNC(width, type) \ + static inline int \ + bus_space_poke_##width(bus_space_tag_t tag, \ + bus_space_handle_t hnd, bus_size_t offset, type value) \ + { \ + bus_space_write_##width(tag, hnd, offset, value); \ + return (0); \ + } +BUS_POKE_FUNC(1, uint8_t) +BUS_POKE_FUNC(2, uint16_t) +BUS_POKE_FUNC(4, uint32_t) + #endif /* KCSAN && !KCSAN_RUNTIME */ #endif /* _X86_BUS_H_ */ _______________________________________________ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
