As digitization continues to transform the heavy-duty vehicle and equipment industry—seen through increased use of telematics, electronics and automation—it is having an effect on the components within the operator cab, specifically the human machine interface (HMI).
“With more digital and connected technology, a far greater amount of information is available from the machines which is useful to the operator,” says Jason Roycht, Vice President and Regional Business Unit Leader, Commercial Vehicles & Off-Road, Bosch. “We must think about how to best organize the information with contextual purpose.”
Danfoss Power SolutionsOne result of this has been the increased development of multifunctional displays. “Our challenge is to simplify the interface in a more complex set of communication and control,” he says. “This helps us improve the traditional information needed, [and] also helps future-proof the interface.”
Aaron Becker, Market Development Manger at Danfoss Power Solutions, says the digital transformation is affecting how operators interact with a machine, and the type of feedback which can be provided in-cab through displays and other HMI components. “As such, the operator is going to be consistently interacting with the electronics of the system at a degree typically higher than is seen in the industry,” he says. “How the cab is designed and the display is laid out so the operator can achieve an easy interaction with their HMI is going to be really important.”
He notes the example of some newer cars today which include a pure touch display for all vehicle feedback. While it demonstrates a nice simplification of the technology on the vehicle, managing multiple screens and interfaces on a single display is distracting and difficult for drivers. “That feedback and interaction with the operator is a really important part of how to make the operator environment easy to be comfortable with and achieve a high level of productivity.”
To overcome this issue, Becker says Danfoss offers components such as its HMK keypad and HMR dial, which complement displays such as its DM product, minimizing the need for the operator to interact directly with the display. Instead they use the keypad or dial, often placed near the machine’s work joystick, to move through the various points of information on the display, reducing distraction. This also reduces the amount of time the operator’s hand is not on the joystick, increasing safety and productivity. “The amount of time they can keep their hand in a working position is going to be heavily related to their productivity level,” he says. Bosch
Enrique Busquets, Engineering Manager, Bosch Rexroth, says the digital transformation has influenced how the operator interfaces are designed to accommodate a higher degree of components’ and systems’ electronification. Hydraulic components are increasingly becoming electronically controlled and connected, which enables automation, improves the entire system efficiency and allows complexity to be handled by software. “Electronic joysticks such as the 4THE5, displays like the DI4 and connectivity units such as the iTraMS allow the operator to have all of the machine functions and analytics at the tip of their fingers whether they are inside or outside of the machine,” he says.
This electronic control of componentry will disrupt the entire off-highway market and continue to transform the operator interfaces, says Busquets. “A higher degree of components being electronically controlled is the key to the revolutionized future of the off-highway market as well as how the operator will control the machine. For Bosch Rexroth, this is already the case in products such as the A4VG Series 35 and the A21VG, which are fully electronically controlled.”
“The technology shift that comes with digitalization and automation forces designers to really conceptualize the entire product experience of the user/operator in the future,” say Volvo Construction Equipment’s (Volvo CE) Patric Klacksell, Director Cab & Operator Environment, and Gustavo Guerra, Design Director. “The focus has shifted from a specific function or feature to a more holistic approach, where the entire interaction is more relevant, while the efficiency and well-being of the operator relies on how all of these touch points come together in the end.”
Klacksell and Guerra say it’s crucial to keep in mind the connectivity among vehicles and within the cab when developing HMI for today’s digitized vehicles and equipment. “The flow of information becomes paramount, as the operator is able to filter only the most important data to perform the task at hand, reducing stress and increasing efficiency.”
The consumer influence
Mark Torbett, Vice President of Sales and Marketing at GS Global Resources Inc., says the automotive industry’s move toward touchscreens and smartphone connectivity have increased similar in-cab experiences within the off-highway market. “Demand for active touchscreens that interface seamlessly with machine controls has never been greater and will continue to expand into new areas of technology-enabled functionality,” says Torbett. “All of this is possible as technology cost within machine control systems is justified with gains in productivity and safety.”
Dr. Michael Ruf, Executive Vice President of Interior Division Business Unit Commercial Vehicles and Aftermarket at Continental, agrees that the consumer industry is having an influence on the heavy-duty vehicle industry. “HMI in vehicles are being increasingly dominated by trends in the consumer market: display-based solutions and the associated interactive capabilities are increasingly taking hold in vehicles.” Because of this, he notes the necessity of developing HMI solutions which can be intuitively and easily controlled by a driver while minimizing distraction and also allowing use of mobile devices.
“Hard-mounted displays are at risk of being replaced by mobile devices,” adds Kevin M. Malliet, Key Account Manager at Hydro Electronic Devices Inc. (HED). He says the company is examining ways to decouple the interface from the vehicle through the use of a tablet or phone-like device. Doing so would enable the operator to control non-safety related functions outside the cab. “It would also allow them to take data collected during operation with them for offline analysis.”
HED’s CANect View, for instance, allows operators to connect a WiFi enabled device, such as a phone or tablet, directly to the vehicle to become an external, mobile operator interface, says Malliet. “With 4G cellular connectivity, customers using HED CL-T17 telematics devices are able to connect directly with the vehicle in any service area from anywhere in the world.” Hydro Electronic Devices Inc.
He says HED is actively developing features and functionality into its displays similar to automotive infotainment systems. Among these are better graphics, multi-video imaging and the use of projected capacitive (PCAP) touchscreens—commonly used in smartphones and tablets. “People expect a digital interface to function like their phone or automobile. Features like intuitive navigation, multi-touch, and quickly digestible data presentation from other sources, maps or cameras” lead to improved operator efficiency and safer vehicle operation.
Having the machine’s HMI perform similarly to that of an operator’s car or smartphone eases the learning curve and helps increase proficiency, and thus safety and productivity, because the person is already familiar with how the technology works. This helps to make training time faster while also minimizing the number of mistakes that might otherwise be made when operating a machine.
In addition, advanced systems such as object detection can more easily be incorporated due to the increased use of electronics and software. Bosch says its 15-in. (38.1-cm) freely programmable cluster “enables future safety and automated solutions as it is easily integrated with advanced driver assistance technologies.” Roycht says driving and maneuvering aids can be incorporated, from contextual video with overlays to simple proximity warnings. “This additional information, when paired to the respective sensors, can provide assistance to the operator.”
He says a multipurpose display is the most elegant and forward-tech proven solution for getting sensor and machine information to an operator. “We don’t want to have to add a new light or gauge for every sensor.
“Bosch believes connectivity and digitalization can bring great benefit to many areas, including off-road,” he adds.
Dr. Ruf says the HMI of the future will “take the form of an overarching, cross-system approach that channels the increasing flow of information resulting from digitalization.” The ideal HMI, he says, only gives the driver the amount of information necessary for the specific driving situation. “In the future, hardware and software components will dynamically adapt to the driver's preferences and needs, as well as to the vehicle's situation and environment. The ultimate aim of such a so-called ‘holistic HMI’ is to make commercial vehicles safer and more intuitive to drive.”
He says this approach will influence current vehicle architectures and make it possible to merge previously discrete products. “From an independently operating freely programmable instrument cluster (FPIC) and an infotainment system located in the center console, it will be possible in the future to control multiple monitors or display areas from a computerized central HMI-ECU (head unit).”
Effects of electrification and automation
The increasing trends of vehicle electrification and automation are an important aspect of the digital transformation and changing operator interface designs. With electrification comes the need for additional information provided to operators, such as battery life, while avoiding information overload to the operator. “The most direct effect that we’ll likely see is how to optimize battery or vehicle operating range, especially with a purely battery electric vehicle,” says Danfoss’ Becker.
This will take the form of the operator interface providing support to maximize the working range, and also give the operator a level of understanding of time available or what work can be done with the existing battery pack. “That interaction in [the] operator interface with battery electric vehicles is going to become a significant, but hidden feature in many of those machines.”
Joacim Wenna, Vice President, Driver Information Systems, Stoneridge Inc., says because electric vehicles are increasingly able to share more vehicle information to the driver, it is driving the need for a digital interface. “Our customers are demanding flexibility with their displays and driver interfaces to allow them to easily adapt to future technologies and trends, such as differing data needs related to different propulsion systems. For example, things like analog rpm gauges will become obsolete. We are also tasked to visually display information in a way that makes sense for the user experience and driver safety.”
Malliet notes that electrification demands sensitivity to power consumption of the interface devices as they will draw on the power source which may have more limitations on available power compared to conventional gas- or diesel-powered equipment. “While user expectations push toward more graphics, larger screens, and actionable data to replace service alert lights, the interface needs to adjust to energy availability,” he says. “Adjustment can be automatic/dynamic, like dimming the display as energy drops, or it could empower the operator to make selective adjustments based on operating conditions.”
More streamlined touch displays are, and will continue to be, common in electric and automated vehicles. OEMs developing these vehicles want to integrate as many functions in as small of an area as possible, says Grayhill Inc.'s Jim O’Donnell. “The vehicle’s CANbus infrastructure allows access to more vehicle sensor [and] status data than ever before.” Use of components such as a multifunctional display makes it possible to provide all of this data in a single device. As noted by the other companies, the challenge will remain in presenting the data in “the most intuitive manner possible so the user doesn’t get lost in information overload,” he says.
O’Donnell says that like the electric vehicle market, autonomous vehicles will need a minimalistic yet efficient user experience while also ensuring the device is capable of surviving use in harsh operating environments.
Stoneridge Inc.During this year’s IAA Commercial Vehicles show, Stoneridge exhibited its FCD4, a conceptual digital driver environment designed to demonstrate what the future driver workplace could look like. “For this year’s IAA show we incorporated a large screen showcasing how the cockpit environment for the driver would look when applying future technology to enable augmented reality in a traffic scene around the truck,” says Wenna. “Specifically, we are showcasing how other types of technology, such as V2X communication and high-performance cameras with object recognition, can increase safety around the vehicle and improve the driving tasks for the driver. These features are all part of the technology offering provided by Stoneridge as well as the HMI control and displays providing the information to the driver.”
With these capabilities, the end goal of have a perfectly maneuvered, safe and fully autonomous truck can be achieved. Regardless of whether there is an operator in the vehicle or a remote location supervising several vehicles, Wenna says the systems in the truck will gradually become more advanced and automatic. “It is important to provide the right information precisely timed for the driver/supervisor to instantly be able to take control in case of a critical situation.”
Wenna says the FCD4 can be applied to off-highway equipment, as well, though the needs are slightly different depending on the application—in mining the ability to ‘see’ other equipment is important while in construction precise maneuvering is the goal. However, the need for vision-based technology and display solutions are significant in both industries.
While the use of multifunctional displays has increased with the industry’s digital transformation, so has the number of displays within an operator cab. Wenna says this has also increased the need for precise operator interface/UI. “It is essential to coordinate the display content,” he says, driving the need for centralization of data processing. “This processing will be done in so-called HMI domain controllers that are connected to the various displays,” he explains. “This centralization actually then defines new products in the vehicle that may cross boundaries of product lines within companies or even between companies.
This technology-driven trend is therefore affecting the supplier base and also drives consolidation of suppliers and new organizational structures at the OEMs,” Wenna adds. “We are already seeing consolidation of companies and new players in the field due to the above trends.”
As OEMs look to create more streamlined interiors, certain HMI components have begun to disappear from the operator cab, such as the steering wheel—and is a trend that is expected to continue. “Gauge clusters are being eliminated at a very fast pace,” says Torbett. “Independent sub systems are going away (ie. transmission shifter and controller) and being integrated into the primary controls platform instead.”
Bosch’s Roycht says it's highly probable dedicated gauges and warning lights will disappear. “It is better to provide contextual information, perhaps even video feeds, based upon the operating conditions at the time,” he explains. “We are already seeing this trend in modern passenger cars that use displays for controls, video, and information. Trends show a similar path lies ahead on equipment.”
Malliet says this trend can already be seen in agricultural equipment where over the past few years physical switches, pedals, steering wheels and analog dashboards have been replaced with in-cab monitors with virtual switches, speed controls and digital readouts of a machine’s current state.
“Now introduce autonomous operation where an operator can look beyond the windshield and perform broader tasks on the display such as viewing training videos, reading maintenance manuals, planning out the next job or even managing other resources needed to finish the job, all while still monitoring the health and state of the machine in real time,” he says. “The interface becomes a monitoring tool and workstation which empowers a more productive, lower cost workforce.”
At its media event in early 2018, ZF Friedrichshafen AG demonstrated its vision for future autonomous vehicle cab environments aimed at providing a high level of occupant safety. ZF’s cockpit of the future concept consists of an interior without a steering wheel, pedals or rearview mirror. Instead, touchscreen displays and a joystick in the middle console are used for the main driving functions of the vehicle. A screen is located in front of each of the passenger seats so either person can control vehicle functions, and information can be switched from one screen to the other. ZF said the concept can be applied to any vehicle with up to level 4 autonomous driving capabilities and allows manual driving when necessary.
Removing the steering wheel and pedals helps simplify the interior design, as well as improves visibility. The joystick in the middle console is used in place of the steering wheel and pedals for functions such as braking. Pushing the joystick forward will engage the vehicle brakes; ZF said moving it forward is the safest method because the body moves forward when coming to a stop, so it’s a more natural action to complete.
“The real benefit of new technologies is that one can really rethink the entire environment focusing on how it should be used, rather than being limited by existing standards,” say Klacksell and Guerra. “Replacing mechanical interface components with digital ones is low hanging fruit, but entirely new modes of control/interaction may arise from the advent of new tech.” Continental
Dr. Ruf says one possible way to organize the way information is being presented in an autonomous vehicle cab is the subdivision of the cockpit into three different areas, each of which has specific functions. “Firstly, there is an area in the windshield that contains all the information relating to automated driving. The second area, located at the level of the steering wheel, displays all the relevant vehicle information – for example camera images, vehicle state and diagnostic data. To the right of this, there is an area for logistical data in which the driver can perform tasks during periods of automated driving.”
An important aspect of developing HMI for autonomous vehicles will be the prioritization of data from the machine to the vehicle operator. Dr. Ruf also says the way in which the driver is involved and informed plays an important role for the acceptance of automated driving; in the "automated driving" mode, it is particularly important that the driver recognizes the vehicle status and understands the behavior of the system as intuitively as possible. When driving operations are taken over by vehicle systems, the question arises of how alert and concentrated truck or coach drivers have to be or how they can perform other useful activities, such as daily planning or office work. Continental is currently working on the necessary HMI, and other systems, for highly automated commercial vehicles with Knorr-Bremse.
As autonomous vehicle development continues, it is anticipated that even the operator cab itself may be removed—a concept demonstrated in recent years by Case IH and Komatsu. However, HMI manufacturers say there will still be a need for their component in some manner.
“Whether there is a cab or not, the basic technology for sensing and operation are still needed,” says Roycht. “It’s highly likely that even the most automated vehicles will still need basic human controls. There will be many cases where an operator must take control. This might be best accomplished with a basic cab or some type of human interaction.”
Displays and other componentry will still be required to monitor and take over control of machinery when necessary, whether that be from a supervisory location or directly on the vehicle.
“While the autonomous vehicle concepts eliminate the vehicle cab, there is still a need for rugged, multifunctional compact user interfaces on these vehicles,” concludes O’Donnell. “Companies that manufacture HMI must envision how the next generation of user will interface with these vehicles to help designers solve their human interface needs.”