Skip to content
Snippets Groups Projects
  1. Apr 19, 2023
  2. Apr 18, 2023
  3. Apr 06, 2023
    • James Morse's avatar
      README: Add list of KNOWN_ISSUES · 8cde8d16
      James Morse authored
      
      This branch has known issues, here is the list.
      ... It may also have unknown issues ...
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      8cde8d16
    • Hesham Almatary's avatar
      MPAM: Fix calculating the bandwidth granularity · 4c871ce2
      Hesham Almatary authored
      
      The minimum bandwidth granularity is calculated from
      MPAMF_MBW_IDR.BWA_WD, which represents the number of bits for
      a fixed-point fraction (See Section 11.3.8 MPAM Reference Manual).
      
      Right now, the calculation works fine for 1, but
      is wrong for 2 bits and more. For example, 1 bit will equal 50%.
      In the current implementation, 2 bits will equal 33% instead of 25%.
      Similarly, 3 bits will equal 25% instead of 12.5%.
      This commit corrects the calculation.
      
      Signed-off-by: default avatarHesham Almatary <hesham.almatary@huawei.com>
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      4c871ce2
    • James Morse's avatar
      arm_mpam: Add debugfs entries to show the MSC/RIS the driver discovered · 5ebc753c
      James Morse authored
      
      Not all of MPAM is visible through the resctrl user-space interface.
      To make it easy to debug why certain devices were not exposed through
      resctrl, allow the properties of the devices to be read through debugfs.
      
      This adds an mpam directory to debugfs, and exposes the devices as well
      as the hierarchy that was built.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      5ebc753c
    • James Morse's avatar
      debugfs: Add helpers for creating cpumask entries in debugfs · 968212e0
      James Morse authored
      
      debugfs has handy helpers to make a bool, integer or string available
      through debugfs. Add helpers to do the same for cpumasks. These are
      read only.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      968212e0
    • James Morse's avatar
      drivers/perf: Add PMU to support reading resctrl counters via perf · 611d9a95
      James Morse authored
      
      Resctrl exposes the cache opccupancy and bandwidth counters to user space
      via files. Each control and monitor group has a copy of these files, and
      can read the counter by reading the file.
      
      MPAM needs to allocate a hardware monitor to do the counting work, and
      there may not be enough for every control and monitor group to have one.
      MPAM needs some indication of which counters are currently in use, it
      needs to be able to reject additional users if there are not enough
      monitors.
      
      Add a PMU driver for resctrl. This never touches the hardware directly,
      it only uses the resctrl API to retrieve the counter values. The PMU
      supports the same events as resctrl. The event parameters are the domain
      id, which user-space already understands, and the control or monitor
      group id. The group id is a 64bit field provided by resctrl that can
      be used in the kernel to retrieve the group. Currently this encodes
      the CLOSID and RMID.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      611d9a95
    • James Morse's avatar
      FIXME arm_mpam: Make mpam_msmon_read() safe for irq-masked callers · 8ada8f11
      James Morse authored
      
      The resctrl_pmu driver calls resctrl_arch_rmid_read() from interrupt
      context. This eventually ends up in mpam_msmon_read() which cross-calls
      to access remote cache-slices, and sleeps if the monitor returned
      not-ready.
      
      To support resctrl_pmu, remove these behaviours if irqs are masked.
      
      Instead of cross-calling, return an error. This means the resctrl_pmu
      can't be supported on platforms with cache-slices that aren't accessible
      from every CPU. Instead of sleeping when the counter is busy, return
      -EBUSY. The resctrl_pmu driver will not update the counter in this case.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      8ada8f11
    • James Morse's avatar
      arm_mpam: Allow MBWU counters to be used, even when not free running · 4a48cc3f
      James Morse authored
      
      Resctrl exposes the cache opccupancy and bandwidth counters to user space
      via files. Each control and monitor group has a copy of these files, and
      can read the counter by reading the file.
      
      MPAM needs to allocate a hardware monitor to do the counting work, and there
      may not be enough for every control and monitor group to have one. Currently
      MPAM doesn't expose the MBM/MBWU counters to resctrl unless there are enough
      monitors.
      
      To allow perf to read these counters via a PMU driver, add support for
      allocating a monitor. To prevent these appearing in the resctrl filesystem,
      report false from resctrl_arch_event_is_free_running() for the MBM event
      types.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      4a48cc3f
    • James Morse's avatar
      arm_mpam: Allow the maximum partid to be overridden from the command line · 8c12e1cd
      James Morse authored
      
      MPAMs bandwidth monitors are only available via resctrl if there are
      enough monitors for each combination of partid and pmg to have one.
      
      As it is unlikely anyone built that many monitors, allow the
      maximum partid the system will use to be set from the kernel
      command-line.
      
      With this, it should be possible to bandwidth monitors to be
      enabled by reducing the number of partid in use.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      8c12e1cd
    • James Morse's avatar
      untested: fs/resctrl: Add support for assigning iommu_groups to resctrl groups · f176cb8c
      James Morse authored
      
      Arm's MPAM has support for assigning devices behind an IOMMU to a
      control or monitor group. This can be used for device-passthrough
      for a VM, or user-space drivers using VFIO to ensure the device
      is either in the same control group as the CPU threads.
      Alternatively, the iommu_group may be assigned to a different
      control group with preferential schema values.
      
      Extend the resctrl tasks file to include iommu_groups. These
      appear as 'iommu_group:0', where 0 is the group number that
      can be found from /sys/kernel/iommu_groups/. iommu_groups
      can be moved between resctrl groups by writing this string
      in the same way as tasks are moved.
      No state is preserved by resctrl, an iommu_group that disappears
      will no longer be listed as being part of a resctrl group. A new
      iommu_group will appear in the default group.
      
      Add helpers to list and move iommu_groups. Architecture specific
      helpers are used to apply the closid/rmid to the iommu_group due
      to the way MPAM emulates CDP.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      f176cb8c
    • James Morse's avatar
      x86/resctrl: Add stubs for the IOMMU helpers · 4d12a8df
      James Morse authored
      
      Arm's MPAM has support for assigning devices behind an IOMMU to a
      control or monitor group. Helpers are needed because of the way
      MPAM emulates CDP.
      
      Add stub versions of the arch helpers used to do this. This allows
      the code to depend on IS_ENABLED(), allowing the compiler to check
      that code on platforms that don't enable the feature.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      4d12a8df
    • James Morse's avatar
      arm_mpam: resctrl: Add iommu helpers to get/set the partid and pmg · 1ed8d78a
      James Morse authored
      
      SMMU that support MPAM can be configured to use a particular partid
      and pmg for a stream. The assignment of an iommu_group and its
      corresponding streams should be done via resctrl.
      
      Add helpers similar to setting a closid/rmid on a task. We need
      the same shifting if the CPUs are using CDP. The SMMU only takes
      one partid, conceptually its always making data accesses.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      1ed8d78a
    • James Morse's avatar
      kobject: Add kset_get_next_obj() to allow a kset to be walked · 792bcc42
      James Morse authored
      
      To expose iommu_groups via the resctrl filesystem, the resctrl driver
      needs to be able to walk the list of iommu_groups. These are exposed
      via sysfs as a kset.
      
      Add kset_get_next_obj() to allow resctrl to walk the kobjects in the
      kset.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      792bcc42
    • James Morse's avatar
      iommu: Add helpers to walk iommu_group and access corresponding ops · 9605ad16
      James Morse authored
      
      ARM SMMU with MPAM support are able to mark streams of traffic with
      the QoS labels MPAM uses. The user-space interface for MPAM is the
      resctrl filesystem, which allows threads to be moved between groups,
      its natural to do the same for iommu_groups.
      
      The resctrl interface lists threads, so will also need to list
      iommu_groups, it will be necessary to walk the list of iommu_groups.
      To ensure this matches what user-space sees via sysfs, it is best
      to walk the kobjects.
      
      Add iommu_group_get_kset() to allow resctrl to retrieve the set of
      iommu_groups.
      
      Split the kobject-to-group code out of iommu_group_get_by_id() and
      expose it as iommu_group_get_from_kobj(), to allow resctrl to get
      the iommu_group from the kobject it already has when walking.
      
      Finally, add iommu_group_get_ops() to allow the iommu ops for a
      group to be retrieved. The MPAM driver will use this with the
      iommu_group from resctrl to call the get/set methods provided by
      the iommu.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      9605ad16
    • James Morse's avatar
      untested: iommu/arm-smmu-v3: Add mpam helpers to query and set state · bd13f847
      James Morse authored
      
      To allow an iommu_group to be moved between resctrl groups as if it
      were a CPU thread, the mpam driver needs to be able to set the partid
      and pmg for the iommu_group.
      
      Use the properties in the STE, as these only apply to one stream.
      
      The MPAM driver also needs to know the maximum partid and pmg
      values that the SMMU can generate. This allows it to determine
      the system-wide common supported range of values. Add a helper
      to return this id register.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      bd13f847
    • James Morse's avatar
      untested: iommu/arm-smmu-v3: Register SMMU capabilities with MPAM · 913f8bb0
      James Morse authored
      
      Traffic in the system can be tagged with a PARTID and PMG. Different
      requestors can support a different number of bits for these fields.
      
      Before MPAM can be used, the MPAM driver has to discover the minimum
      number of bits supported by any requestor, which affects the range
      of PARTID and PMG that can be used.
      
      Detect whether the SMMU supports MPAM, if it does provide the MPAM
      driver with the maximum PARTID and PMG values.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      913f8bb0
    • James Morse's avatar
      fs/resctrl: cgroup: Add resctrl cgroup controller · 78c662d6
      James Morse authored
      
      Resctrl allows tasks to be grouped into control groups, (a totally
      separate interface to cgroups), each of which have a configuration
      policy that affects the tasks prioritisation in the cache, or the
      memory bandwidth available when the task is running.
      The resctrl tasks file is used to assign threads to a control group.
      New threads inherit the resctrl settings from their parent.
      
      If a process is continually creating new threads, it can create new
      processes while user-space is assigning the threads to the control
      group. This means user-space has to continually poll the list of threads
      when it wants to move a process between control groups.
      
      Another level of abstraction would help, so that a group of threads
      can be moved between control groups in one go. New threads should
      inherit the new value, even if their parent hasn't been updated yet.
      
      Add a new cgroup controller named resctrl. This allows a cgroup to be
      labelled with the 'id' of a resctrl control or monitor group, which
      is used to look up the closid and rmid.
      New processes inherit the setting from the cgroup, meaning that new
      processes created after the label is changed use the new label.
      The relative path of the resctrl group is provided for convenience.
      
      TODO: turns out cgroups now supports threads, how does that work?
      TODO: migration is a thing in cgroups - what is that about?
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      
      N.B. This uses the id for writes in preference to the path as perf
      is already restricted to a u64 for the configuration.
      78c662d6
    • James Morse's avatar
      fs/resctrl: Export the closid/rmid to user-space · 0b2b277d
      James Morse authored
      
      Control and monitor groups have a CLOSID and/or RMID that is used to
      count the cache usage and memory bandwidth of tasks in this group.
      
      Not all of MPAMs counters can be exposed via resctrl, as each counter
      also needs a monitor to be allocated. It is unlikely there are enough
      monitors for every RMID to have a monitor permanently allocated.
      
      To allow counters to be read via perf, the RMID that a control
      or monitor group is using needs exposing to user-space. This can be
      passed back to perf as a parameter. MPAM's PMG values are not
      unique, the PARTID needs to be provided too. Perf allows a number of
      u64 arguments, which is not enough to encode a control/monitor group
      name.
      
      Similarly, there has been some interest in allowing cgroup to manage
      the tasks file for resctrl. Exposing a unique identifier for each
      control or monitor group will allow cgroups to point to a resctrl
      group that holds its configuration.
      
      Provide a file in each control or monitor group that returns a unique
      identifier. When passed back to the kernel, resctrl can decode this
      into a closid/rmid, or just identify the control or monitor group.
      
      The value is xor'd with a value picked at boot as obsfucation. This
      is to prevent user-space from relying on the layout of this field,
      or re-using values between boots of the system. This is to allow the
      kernel to change the layout of this field in the future.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      0b2b277d
    • James Morse's avatar
      fs/resctrl: Add this_is_not_abi mount option · 42fc5a24
      James Morse authored
      
      Some later things in the MPAM tree enable behaviour that resctrl doesn't
      have upstream. To make it clear to people using the out-of-tree code that
      they shouldn't be relying on this in user-space, add a mount option to
      enable this stuff.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      42fc5a24
    • James Morse's avatar
      arm_mpam: resctrl: Update the rmid reallocation limit · 914331c7
      James Morse authored
      
      resctrl's limbo code needs to be told when the data left in a cache is
      small enough for the partid+pmg value to be re-allocated.
      
      x86 uses the cache size divded by the number of rmid users the cache
      may have. Do the same, but for the smallest cache, and with the
      number of partid-and-pmg users.
      
      Querying the cache size can't happen until after cacheinfo_sysfs_init()
      has run, so mpam_resctrl_setup() must wait until then.
      
      Signed-off-by: default avatarJames Morse <james.morse@arm.com>
      914331c7
Loading