A portable audio-haptic-coupled paper-based device seamlessly integrating passive learning for effortless skill acquisition
0 Abstract
Integrating passive haptic learning (PHL) into paper-based devices offers significant potential for enhancing user immersion and skill acquisition. Herein, we present a compact audio-haptic-coupled paper-based device constructed from flexible, readily available, and eco-friendly materials. The device generates mechanical vibration under low-frequency excitation (~200 Hz), serving as a PHL indicator that can be easily sensed by humans, while higher-frequency excitation (>1,000 Hz) enables the production of diverse musical scales. The device achieves a sound pressure level of 70 dB, which is clearly perceived by audiences. Moreover, it exhibits excellent fatigue resistance, displaying negligible performance degradation even after 43,200 cycles of continuous compression. To improve portability, two compact circuit boards enabling the device to operate in various environments were developed. In practice, volunteers demonstrated improved learning efficiency when using the device integrated with PHL, achieving a ~35% reduction in learning time and 100% performance accuracy.
Keywords
INTRODUCTION
Acoustic devices capable of transmitting sound information and providing immersive experiences hold considerable potential in human-machine interfaces[1-8] and augmented/virtual reality[9-11] applications. Conventional acoustic devices, such as microphones and loudspeakers, typically incorporate bulky and rigid components, which can cause discomfort during prolonged use. To overcome this issue, researchers have increasingly explored lightweight and flexible materials[12-16], such as paper, for the development of next-generation acoustic devices[17-20]. Paper, being abundant and biodegradable, offers several notable advantages, including low cost, environmental friendliness, lightweight, and mechanical flexibility. Additionally, paper-based acoustic devices demonstrate stable frequency response characteristics and superior sound quality[21-27]. Given these advantages, a variety of paper-based acoustic devices have been proposed. For instance, a paper-based loudspeaker that integrates seamlessly behind curtains was developed, replacing bulky conventional devices while delivering clear and immersive audio[23]. Similarly, Su et al. reported a paper-based sound detector that can “visualize” acoustic signals[21]. Although significant advancements have been achieved, most existing paper-based acoustic devices provide only basic auditory experiences by simply reproducing recorded sounds. Unfortunately, this approach limits user engagement and interactivity, hindering creativity in audio interaction. Audio devices that allow free-form playing (e.g., musical instruments) are powerful media for expressing emotions, capable of producing a wide variety of auditory effects through combinations of different notes. For instance, short and light notes often convey joy, while prolonged, lower-pitched notes typically express sadness[28-33]. However, mastering such instruments is a gradual process that requires repetition and training over time. As a promising solution, passive haptic learning (PHL) technology enables users to acquire skills through subconscious haptic stimulation with minimal conscious effort, following a “caught rather than taught” approach[34-37]. Recent works also demonstrate the effectiveness of PHL-assisted wearable devices in enhancing skill acquisition. For instance, exoskeleton/vibration gloves equipped with motors, delivering haptic feedback through repetitive vibration sequences, are widely used in Braille typing and piano education applications[38-41]. However, as external auxiliary equipment, existing PHL-assisted wearable devices require coordination with additional audio equipment for operation. The bulky, rigid, and venue-specific characteristics of these devices significantly impact usage scenarios. Therefore, seamlessly integrating PHL systems into musical instruments, especially fragile paper-based ones, remains a highly desirable but challenging task.
In this work, we present an audio-haptic-coupled paper-based device seamlessly integrated with PHL, mainly consisting of flexible, readily available, and eco-friendly materials, including polylactic acid (PLA) electret film and paper substrates. The devices obtained can effortlessly convert electrical signals into mechanical vibrations at low frequencies, which serve as the PHL indicator. Additionally, this device can produce various musical scales for playing at high frequencies. To improve portability, two compact circuit boards were developed to drive the device.
EXPERIMENTAL
Fabrication of the paper-based device
The detailed fabrication processes of the device consist of six steps, as shown in Supplementary Figure 1.
Step I (laser cutting): A laser cutting machine (4060 70W, Ming Chuang Co., China) was used to cut paper substrates (30 μm: tracing paper, Tmall Youtai Office Store, China; 50 μm: rice paper, Tmall Youtai Office Store, China; 130 μm: printing paper, Tmall Youtai Office Store, China; 400 μm: cotton paper, Tmall Youtai Office Store, China) into various shapes, such as squares, T-like shapes, and irregular shapes.
Step II (folding): The irregular paper was folded by bending its two lateral sides inward to form two arcs that serve as an elastic support frame. After folding, the support forms a square with sides measuring 2 cm in length.
Step III (fabricating electrodes): Ag (Shenzhen Jingzhe Technology Co., LTD, China) was uniformly sprayed onto the paper substrates to form the Ag/paper electrode layers.
Step IV (adhering PLA film): PLA (Shenzhen Esun Industrial Co., Ltd., China) electret films were attached onto the top electrode of the haptic feedback unit and the middle electrode of the loudspeaker unit. The interfaces between the PLA films and the paper substrates were secured using double-sided tape.
Step V (corona charging): The PLA films were placed facing the corona needle at a distance of 5 cm, corona charging was applied using a direct current (DC) power supply (DW-N503-4ACD2, Tianjin Dongwen, China) at 16 kV for the positively charged middle electrode of the loudspeaker unit and -16 kV for the negatively charged top electrode of the haptic feedback unit for 5 min, resulting in stable surface potential values of approximately +310 and -320 V, respectively.
Step VI (assembling): The final device was assembled by connecting the haptic feedback unit, loudspeaker unit, and the support frame. All boundaries were fixed with double-sided tape.
Characterization
This device was powered by a function generator (33210A, KEYSIGHT, USA) and amplified 100× [E-464, Physik Instrumente (PI), Germany; ATA-7010, Aigtek, China]. The vibration displacement of the haptic feedback unit was measured by a laser Doppler vibrometer (LDV) (Vibro One, Polytec, Germany), and the displacement data was collected by a data acquisition system [National Instruments (NI) USB-6341, USA]. The sound pressure level (SPL) data was evaluated by a multifunctional noise analyzer (AWA6228+, AIHUA, China) under a background noise level of about 35 dB. The surface potential values of the PLA electret films were recorded using an electrostatic voltmeter (Trek 347, USA). Scanning electron microscopy (SEM) images were acquired using a high-resolution field emission scanning electron microscope (Sigma FE-SEM, Zeiss Corporation, Germany).
Volunteer tests
Tests with volunteers were conducted in compliance with ethical regulations under a protocol approved by the ethical committee of the University of Macau (approval number BSERE21-APP022-FST). All volunteers provided written informed consent before participating in the tests.
RESULTS AND DISCUSSION
Design strategy and fabrication
Sound can fundamentally be defined as the auditory perception of vibration. Following this principle, we designed and fabricated a paper-based device based on PLA films. As a superior electret material, the PLA film can provide mechanical deformation in response to electric fields, resulting in sound (loudspeaker) and vibration (haptic feedback). The lightweight and flexible properties of PLA films enhance seamless integration and reduce the overall weight of the device. As illustrated in Figure 1A, the device includes two primary components: a haptic feedback unit and a loudspeaker unit. The haptic feedback unit consists of a single PLA electret film (thickness: 40 μm) sandwiched between top and bottom Ag/paper electrodes. These Ag/paper electrodes were fabricated by spray-coating Ag paste onto paper substrates. The surface SEM images before and after Ag coating are shown in Supplementary Figure 2. Ag exhibits strong adhesion to the surface of the paper substrate without altering the surface morphology. The thickness of the top and bottom electrodes is 30 and 50 μm, respectively. Similarly, the loudspeaker unit includes a single PLA film and three Ag/paper electrodes (top, middle, and bottom electrodes). The thickness of the top, middle, and bottom electrodes is 130, 30, and 400 μm, respectively. To enhance sound transmission efficiency from the interior to the exterior of the loudspeaker unit, all electrodes were perforated [pore diameter: 0.6 mm and density: 20.25 holes/cm2 (D = N/A, where N represents the number of holes, and A denotes the area of the paper]. Furthermore, the haptic feedback unit and loudspeaker unit are separated by a support frame with an origami structure [Figure 1B]. The dimensions and detailed fabrication processes of the device are provided in Supplementary Figures 1 and 3.
Figure 1. Structural design and working principle of the paper-based device with PHL. (A) Application scenario and optical photo (author team members shown; photographed by the authors); (B) Detailed structure, including a haptic feedback unit and a loudspeaker unit; (C) Block diagram showing the working principle of the device; (D) Key advancements and application scenarios. PHL: Passive haptic learning; PLA: polylactic acid.
PLA film retains a large amount of static charge for a long time, exhibiting excellent reliability for both vibration and acoustic performance[42-44]. Supplementary Figure 4A shows the negative and positive surface potentials of the PLA film after 70 days. The surface potentials remain stable at ~-320 and ~+310 V, respectively. Moreover, as an eco-friendly material, PLA film fully degrades in about 200 days at 60 °C and 100% relative humidity (RH) conditions [Supplementary Figure 4B]. It should be noted that the device is powered by two different alternating-current (AC) signals: the loudspeaker unit operates under a continuous signal, while the haptic feedback unit operates using an intermittent signal. Thus, the working process consists of four stages, as shown in Figure 1C. Initially, the fingers lightly touch the surface of the haptic feedback unit of the device (preload < 100 mN). In this relaxed state, the top electrode and middle electrode of the loudspeaker unit remain apart, resulting in an inactive state of the loudspeaker unit. Subsequently, a fixed low frequency of 200 Hz is supplied to the haptic feedback unit to produce vibration, which serves as an indicator. The fingers receive the haptic stimulation and then press down onto the loudspeaker unit. Finally, the top and middle electrodes make contact due to pressure, which activates the sound-generation function of the loudspeaker unit. Depending on the real-time haptic cues generated by the haptic feedback unit, the volunteers who used the integrated device exhibited higher scale recognition accuracy and completed tasks more quickly than those who used a similar device without haptic assistance. Furthermore, since paper is an insulating material, these devices require minimal safety considerations. The main advantages and application scenarios of this device are illustrated in Figure 1D. As expected, compared to conventional skill-acquisition methods, learners using our audio-haptic-coupled paper-based device achieve higher playing accuracy and greater learning efficiency.
Parameter optimization of loudspeaker unit
To optimize the sound output performance, the SPL of the paper-based devices (device size: 2 × 2 cm2) was evaluated. SPL values were collected in a custom-built small soundproof chamber, employing a multifunctional noise analyzer [Supplementary Figure 5]. During testing, the noise analyzer was placed vertically, 2 cm away from the surface of the device. A waveform generator produced the driving signal, which was then amplified via a voltage amplifier [Figure 2A]. All results presented in Figure 2B-I represent the mean ± standard deviation of three identically fabricated devices, unless otherwise specified. As shown in Figure 2B, when the driving voltage increases from 100 to 500 V, the perforated device exhibits higher SPL values than the non-perforated one, highlighting the importance of the perforated structures in enhancing acoustic transmission out of the device. To elucidate the role of perforation geometry, we investigate the effects of hole diameter and density on SPL values [Figure 2C]. As illustrated in Figure 2D, increasing the diameter of the holes from 0.6 to 1.2 mm enhances SPL performance, confirming that larger apertures facilitate more efficient sound propagation. Thus, the best SPL of 90 dB was obtained with a hole diameter of 1.2 mm. Similarly, higher hole density resulted in a higher SPL value [Figure 2E].
Figure 2. Parameter optimization of loudspeaker unit. (A) Schematic representation of an SPL measurement; (B) Results showing the SPL response of perforated and non-perforated paper substrates at different driving voltages. The excitation frequency is 1,975 Hz; (C) Optical photos and schematic diagrams of paper substrates with varying hole dimensions and densities; (D) Effect of hole diameter on SPL; (E) Relationship between SPL and hole density; (F) Effect of the top paper substrate with varying thickness on SPL; (G) Relationship between the thickness of the middle and bottom paper substrates and SPL; (H) PLA electret film with varying thickness as a function of SPL value; (I) PLA electret film with different surface potentials as a function of SPL value. Unless otherwise stated, the test conditions include a driving voltage of 500 V, a hole diameter of 1.2 mm, and a hole density of 20.25 holes/cm2. Error bars in the figures represent the standard deviation for an experimental sample size of three. SPL: Sound pressure level; PLA: polylactic acid.
A series of experiments was conducted to examine the SPL response to excitation frequencies. Figure 2F shows that when the excitation frequency increases from 261 to 1,975 Hz, SPL values for devices with different top-paper thicknesses vary by less than 10 dB, suggesting that top-paper thickness is not a key determinant of SPL output. Indeed, the primary role of the top electrode in the loudspeaker unit is to function as an on-off switch button: the loudspeaker unit is active only when the middle and bottom electrodes are charged simultaneously. Figure 2G demonstrates that SPL value depends strongly on the thicknesses of the middle and bottom paper layers [Supplementary Figure 6]. The experimental results agree well with the simulation model described in our previous work[23]. A thinner middle substrate tolerates greater mechanical deformation, resulting in improved sound penetration, while a thicker bottom substrate maintains overall device stability. Consequently, the optimal configuration, characterized by the thinnest middle-paper (30 μm) and the thickest bottom-paper (400 μm), achieves the highest SPL value (> 90 dB). Furthermore, the SPL-frequency response curve in Figure 2H exhibits a clear increase in SPL as the PLA electret film thickness decreases. This occurs because thinner PLA films undergo larger vibration displacement under the same electrostatic force. Additionally, we explored the impact of PLA surface potential on the SPL. The surface potential is effectively equivalent to an applied DC bias[45], enabling either a reduction in the AC driving voltage requirement or an increase in the SPL output for a given input voltage, as proven in Supplementary Figure 7. Figure 2I shows that devices with the highest PLA film surface potential (~300 V) produce the highest SPL output. After comprehensive parameter optimization, the device achieves significant improvements in SPL output and frequency response characteristics.
Parameter optimization of haptic feedback unit
Generally, under a given preload, the higher the vibration displacement generated by a device under external excitation, the more pronounced the haptic stimulus perceived by the user. To explore the factors influencing vibration displacement in the developed paper-based device, a series of systematic experiments was conducted. As illustrated in Supplementary Figure 8, the vibration displacement was evaluated using a custom experimental setup. In this configuration, lasers from the LDV were directed onto the surface of the haptic feedback unit. The vibration displacement of the device was recorded using an NI data acquisition system, and the collected data was transferred to a laptop for analysis. The paper-based device excitation was driven by a waveform generator and a voltage amplifier. The vibration displacement distribution of the haptic feedback unit is mapped and shown in Figure 3A. Dark blue areas represent regions where haptic stimulation was strongest. The maximum displacement occurred at the center, reaching more than 3 µm. Therefore, unless otherwise stated, all vibration displacement data presented below was collected from the center of the haptic feedback unit.
Figure 3. Parameter optimization of haptic feedback unit. (A) Vibration displacement distribution of the haptic feedback unit, showing that the vibration displacement is highly dependent on the test position; (B) Response curve of displacement to different configurations of the haptic feedback unit. The configurations include the combination of different thicknesses of the top and bottom paper substrates; (C) Relationship between vibration displacement and preload; (D) Frequency response of the haptic feedback unit when driven by sine wave and square wave; (E) Measured displacement as a function of driving voltage; (F) A photo exhibiting the detail of the user perception experiment (author team members shown; photographed by the authors); (G) Recognition accuracy of the paper-based device for each channel. Unless stated otherwise, the test conditions were a driving voltage of 500 V, an excitation frequency of 200 Hz, and a square wave input. The error bars in the figures represent the standard deviation, with an experimental sample size of three.
The effect of paper substrate thickness on vibration displacement was evaluated and is illustrated in Figure 3B. The highest displacement was achieved when the paper thickness was 30 or 50 μm. Figure 3C depicts the vibration displacement of the paper-based device under different preload conditions. In the absence of preload, the device exhibits a maximum displacement of ~2.5 µm. As the preload increases, a notable decline in displacement is observed, reaching ~1 µm at a preload of 100 mN. This finding suggests that users should apply only light pressure on the paper-based device to avoid excessive compression during operation. To identify the optimal preload range, the relationship between preload and compressive strain was analyzed, as shown in Supplementary Figure 9. When the preload exceeds 100 mN, the compressive strain of the paper-based device exceeds 40%. At this level of compression strain, undesirable acoustic noise is generated by the loudspeaker unit. Therefore, to obtain pronounced haptic stimulation and avoid noise interference, the applied preload should be maintained below 100 mN.
Furthermore, the displacement values as a function of excitation frequency (square wave) were measured, as shown in Supplementary Figure 10. A maximum displacement of ~3 μm occurred at ~200 Hz. It should be noted that the displacement response is relatively flat across the full frequency range (100-400 Hz), which corresponds to the frequency range most sensitive to human skin[46]. Figure 3D shows the relationship between the voltage waveform and the vibration displacement. Compared to the sine wave, the square wave demonstrated enhanced vibration performance across all frequencies. The resonant frequency of the paper-based device was recorded at ~200 Hz. The vibration displacement response to the driving voltage is shown in Figure 3E, indicating that displacement increases with increasing driving voltage. Notably, electrostatic adhesion occurs when a positive voltage is applied to the top electrode. Supplementary Figure 11 shows the experimental setup for measuring the displacement of a suspended actuator and that of an actuator experiencing electrostatic adhesion. The measured displacement with electrostatic adhesion is significantly greater [Supplementary Figure 12]. Thus, an excitation frequency of 200 Hz, a driving voltage of 500 V, together with a square wave and electrostatic adhesion, produces the highest displacement (~3 μm). To further evaluate the sensitivity of the haptic feedback unit, a user perception experiment was conducted, as illustrated in Figure 3F. Volunteers wore blindfolds to prevent visual cues, and both volunteers and testers remained silent to avoid verbal prompts. Each volunteer lightly placed their fingertips (< 100 mN) sequentially on seven individual paper-based devices to sense the haptic stimulation delivered to each finger. Each device had a single input-output communication channel. The tester then randomly applied an excitation signal to one of the seven devices to activate it. Each one of the seven channels was randomly activated three times. When a volunteer perceived haptic stimulation from a specific fingertip, they pressed the corresponding device. A correct result was recorded when the volunteer’s selection matched the device stimulated by the tester. To minimize interference from sudden changes in equipment status, the function generator and voltage amplifier remained on throughout the test. The recognition accuracy across all 30 volunteers consistently reached 100% for each channel, as shown in Figure 3G.
Fatigue resistance of the paper-based device
As illustrated in Figure 1B, the top and bottom electrodes of the loudspeaker unit are separated by an origami-structured support frame. When the distance of these electrodes reduces to 40% of its initial value, the device generates noticeable noise [Figure 4A]. This phenomenon emphasizes the importance of ensuring that all components rapidly return to their original state after compression. Fortunately, our paper-based device demonstrates superior fatigue resistance. The thickness of the device decreases by only ~13.8% after 12 h of continuous pressing [Supplementary Figures 13 and 14]. The pressing frequency was once per second. Compared to Supplementary Figure 2B, the post-cycling SEM image of the Ag/paper shown in Supplementary Figure 15 demonstrates that the fatigue test does not degrade the conductive Ag surface layer. Meanwhile, conductivity tests performed before and after the fatigue test further confirm its durability [Supplementary Figure 16]. Thus, the performance of the device - both the sensitivity of the haptic feedback unit and the SPL of the loudspeaker unit - exhibits negligible degradation [Figure 4B]. The displacement of the haptic feedback unit decreases by less than 0.3 μm, and the SPL performance of the device demonstrates a reduction of less than 5 dB after 12 h of continuous pressing. Additionally, compared with a new device, the SPL performance of a device continually pressed for 12 h exhibits less than 5 dB attenuation when the test distance increases from 2 to 15 cm [Figure 4C]. Finally, to assess the sound quality of the device after continuous pressing, we collected the signal-to-noise (SNR) ratio for each musical scale, as shown in Figure 4D. All results indicate the superior fatigue resistance of the paper-based device. To explore the factors influencing the fatigue performance of this device, the experimental scenario was reconstructed using SOLIDWORKS software [Supplementary Tables 1 and 2]. In this simulation, the deformation of the support frame is induced by the pressure exerted by an external cylinder. As illustrated in Figure 4E, the applied pressure is primarily distributed in two regions: around the cylinder and within the folding area. For an equivalent level of deformation, a higher pressure is needed for the non-perforated support frame compared with the perforated frame [Supplementary Figure 17A]. Therefore, the perforation design performs two essential roles in paper-based devices: (i) enabling more efficient sound transmission from the interior to the exterior of the device; and (ii) enabling the device to release stress through the perforated regions. However, because the non-perforated support frame lacks localized weak points, its surface pressure distribution is more uniform. In contrast, the perforated support frame is more prone to fatigue under repeated compression due to local stress concentration around the holes [Figure 4F and Supplementary Figure 17B]. Supplementary Figure 17C shows that fatigue resistance decreases with increasing compressive strain, indicating a strong dependency on extrusion displacement. An optimal fatigue lifetime of ~43,200 cycles is achieved when the compressive strain is ~25%.
Figure 4. Fatigue resistance evaluation of the paper-based device. (A) The relationship between SPL and compressive strain. The compressive strain refers to the distance between the top and bottom electrodes of the paper-based device; (B) The collected displacement of the haptic feedback unit and SPL values of the loudspeaker unit at different times during the continuous pressing of 12 h (~43,200 cycles); (C) Comparison of SPL generated from paper-based device before and after the continuous pressing test; (D) The SNR in distinct excitation frequencies after the continuous pressing test; (E) Simulation of stress distribution for perforated support frame; (F) Comparison of fatigue resistance between perforated and non-perforated support frames. A fixed compressive strain of 30% is applied. The error bars in the figures represent the standard deviation, with an experimental sample size of three. SPL: Sound pressure level; SNR: signal-to-noise.
Demonstration of the portable paper-based device
To demonstrate the portability of the developed device, we propose a hypothetical scenario, illustrated in Figure 5A. In this scenario, seven devices (each measuring 2 × 2 cm2) are integrated into a book to create a paper-based “piano”. The input signals of the seven devices - including both the haptic feedback unit and loudspeaker unit - are controlled by two self-made compact circuit boards [Figure 5B]. The haptic feedback units in all seven devices were supplied with a 200 Hz excitation signal frequency in a programmed sequence. The circuit board driving the loudspeaker unit contains seven output channels, each producing a different excitation frequency corresponding to do/re/mi/fa/so/la/ti. As shown in Figure 5C, the musical score is converted into a sequence of vibrations through programming, with each scale corresponding to one 500-ms vibration, with a 1 s interval between scales. These regular mechanical vibrations provide powerful assistance to players, thus enabling more effective performance. Figure 5D-F displays the performance of the portable device controlled by circuit boards. In Figure 5D, applying a fast Fourier transform (FFT) to the musical scales generated from the device converts the time-domain audio signal into its frequency-domain representation, enabling evaluation of the accuracy of the musical scales. The resulting spectrum shows a single peak per note, indicating the pure-tone behavior of the device. Compared to standard do/re/mi/fa/so/la/ti frequencies, the frequency positions emitted by the device match well, exhibiting almost no peak shift [Supplementary Table 3]. Moreover, the SPL response of the device to different musical scales ranges from 70 to 75 dB [Figure 5E]. Figure 5F shows that the vibration displacement of the haptic feedback unit averages about 2 μm, a level easily perceived by human fingertips. As a demonstration, Twinkle Twinkle Little Star was performed using the device [Supplementary Movie 1]. It is also worth noting that the power consumption of the circuit board is less than 14 mW [Supplementary Figure 18].
Figure 5. Demonstration of the portable paper-based device. (A) Photo illustrating a representative usage scenario of the portable paper-based device (author team members shown; photographed by the authors); (B) Photo showing detailed information on the two portable circuit boards (photographed by the authors); (C) Block diagram illustrating the operational workflow of the paper-based device integrating the haptic feedback unit and loudspeaker unit; (D) Results demonstrating the fidelity of the paper-based device in different musical scales; (E) SPL values corresponding to the different musical scales; (F) Vibration displacement in response to different musical scales, indicating that the haptic feedback unit produces stimuli that are easily perceived by human fingers; (G) The waveforms of Two Tigers performed by volunteers in different groups. The error bars represent the standard deviation, with an experimental sample size of three. SPL: Sound pressure level; MCU: microcontroller unit; GPIO: general-purpose input/output; PWM: pulse width modulation; HV: high voltage; PHL: passive haptic learning.
Supplementary Movies 2 and 3 demonstrate that volunteers can skillfully perform music using the paper-based device, especially when assisted by the haptic feedback unit. In this work, the capability to acquire proficient playing skills through haptic stimulation is defined as PHL. To evaluate its effectiveness, 30 volunteers were divided into three groups: an experimental group and two control groups. All groups were instructed to perform the same musical piece, “Two Tigers”. To avoid proficiency-related bias, each volunteer performed only once. Specifically, volunteers in the experimental group used the paper-based devices integrated with PHL. Control group 1 used the same device but without PHL, while control group 2 also used the device without PHL but were given 30 min of practice beforehand. The performance efficiency and accuracy of the three groups are compared in Figure 5G. The representative data shown in the figure correspond to volunteers whose performance was closest to their group’s average. Briefly, volunteers in the control groups required over 42 s to complete the piece and made errors [Supplementary Movie 2]. Although 30 min of practice improved the accuracy rate in control group 2, errors remained. In contrast, a volunteer in the experimental group performed the same song without prior practice in just 31 s, reducing performance time by 35% while achieving 100% accuracy [Supplementary Movie 3]. This result demonstrates a significant enhancement in skill acquisition efficiency enabled by our device with PHL.
CONCLUSIONS
In summary, we propose a compact paper-based device seamlessly integrated with PHL technology, consisting of a haptic feedback unit and a loudspeaker unit. The device offers advantages such as low cost, light weight, and eco-friendliness. To overcome the limitations of conventional bulky musical instruments, we further developed two compact circuit boards for controlling our device. Future work to improve the performance of the proposed paper-based device could include: (i) optimizing PLA electrets or developing new eco-friendly electret materials to improve surface potential, enabling reductions in driving voltage and power consumption; (ii) enhancing the fatigue resistance of these devices to ensure long-term performance under practical operating conditions and in challenging environments.
DECLARATIONS
Authors’ contributions
Made substantial contributions to the conception and design of the study and performed data analysis and interpretation: Pi, Y.; Gong, Y.
Designed circuit boards: Zhao, D.
Assisted with the experiment: Zhang, K.; Guan, X.; Zhou, Y.
Investigated related works: Dai, N.; Liu, Z.
Directed the research and revised the paper: Gong, Y.; Zhong, J.
Availability of data and materials
The original contributions presented in this study are included in the article/Supplementary Materials. Further inquiries can be directed to the corresponding authors.
AI and AI-assisted tools statement
Not applicable.
Financial support and sponsorship
This work is supported by the National Natural Science Foundation of China (52522513), the Science and Technology Development Fund, Macau SAR (File No. 0117/2024/AMJ), Zhuhai UM Science & Technology Research Institute (CP-009-2024), University of Macau (MYRG-GRG2024-00121-FST-UMDF, MYRG-CRG2024-00014-FST-ICI, MYRG-GRG2025-00038-FST-UMDF), and the Postdoctoral Fellowship Program of CPSF (GZC20231160).
Conflicts of interest
All authors declared that there are no conflicts of interest.
Ethical approval and consent to participate
The tests with volunteers are performed in compliance with ethical regulations under a protocol approved by the ethical committee of the University of Macau (approval number BSERE21-APP022-FST). All volunteers provided written informed consent at the time of recruitment.
Consent for publication
The individuals appearing in the images included in this manuscript are members of the author team. Consent for the use and publication of these images has been obtained from the individuals involved.
Copyright
© The Author(s) 2026.
Supplementary Materials
REFERENCES
1. Park, J.; Kang, D. H.; Chae, H.; et al. Frequency-selective acoustic and haptic smart skin for dual-mode dynamic/static human-machine interface. Sci. Adv. 2022, 8, eabj9220.
2. Xiao, Y.; Li, H.; Gu, T.; et al. Ti3C2Tx composite aerogels enable pressure sensors for dialect speech recognition assisted by deep learning. Nanomicro. Lett. 2024, 17, 101.
3. Kim, J.; Chamorro, L. P. Coupled mechanics in skin-interfaced electronics via computer vision methods. Soft. Sci. 2024, 4, 12.
4. Chen, S.; Ouyang, Q.; Miao, X.; et al. Wearable ultrasound devices for therapeutic applications. Nanomicro. Lett. 2025, 18, 45.
5. Qiao, W.; Zhou, L.; Zhang, J.; et al. A highly-sensitive omnidirectional acoustic sensor for enhanced human-machine interaction. Adv. Mater. 2024, 36, e2413086.
6. Lee, J. H.; Cho, K. H.; Cho, K. Emerging trends in soft electronics: integrating machine intelligence with soft acoustic/vibration sensors. Adv. Mater. 2023, 35, e2209673.
7. Sun, H.; Gao, X.; Guo, L. Y.; et al. Graphene‐based dual‐function acoustic transducers for machine learning‐assisted human–robot interfaces. InfoMat 2022, 5, e12385.
8. Shahbaz, I.; Li, L.; Rehman, S. U.; et al. Deep-learning-assisted tympanic-modeled lead-free rod-based piezoelectric transducer for acoustic sensing. Device 2025, 3, 100801.
9. Kang, D.; Lee, C. G.; Kwon, O. Pneumatic and acoustic suit: multimodal haptic suit for enhanced virtual reality simulation. Virtual. Reality. 2023, 27, 1647-69.
10. Serafin, S.; Geronazzo, M.; Erkut, C.; Nilsson, N. C.; Nordahl, R. Sonic interactions in virtual reality: state of the art, current challenges, and future directions. IEEE. Comput. Graph. Appl. 2018, 38, 31-43.
11. Kim, J. J.; Wang, Y.; Wang, H.; Lee, S.; Yokota, T.; Someya, T. Skin electronics: next‐generation device platform for virtual and augmented reality. Adv. Funct. Mater. 2021, 31, 2009602.
12. Kim, J.; Jung, G.; Jung, S.; et al. Shape-configurable MXene-based thermoacoustic loudspeakers with tunable sound directivity. Adv. Mater. 2023, 35, e2306637.
13. Che, Z.; Wan, X.; Xu, J.; Duan, C.; Zheng, T.; Chen, J. Speaking without vocal folds using a machine-learning-assisted wearable sensing-actuation system. Nat. Commun. 2024, 15, 1873.
14. Keplinger, C.; Sun, J. Y.; Foo, C. C.; Rothemund, P.; Whitesides, G. M.; Suo, Z. Stretchable, transparent, ionic conductors. Science 2013, 341, 984-7.
15. Lin, Z.; Duan, S.; Liu, M.; et al. Insights into materials, physics, and applications in flexible and wearable acoustic sensing technology. Adv. Mater. 2024, 36, e2306880.
16. Yang, G. H.; Lin, J.; Cheung, H.; et al. Single layer silk and cotton woven fabrics for acoustic emission and active sound suppression. Adv. Mater. 2024, 36, e2313328.
17. Yang, L.; Wang, H.; Yuan, W.; et al. Wearable pressure sensors based on MXene/tissue papers for wireless human health monitoring. ACS. Appl. Mater. Interfaces. 2021, 13, 60531-43.
18. Lin, Y.; Gritsenko, D.; Liu, Q.; Lu, X.; Xu, J. Recent advancements in functionalized paper-based electronics. ACS. Appl. Mater. Interfaces. 2016, 8, 20501-15.
19. He, X.; Zi, Y.; Yu, H.; et al. An ultrathin paper-based self-powered system for portable electronics and wireless human-machine interaction. Nano. Energy. 2017, 39, 328-36.
20. Liao, X.; Zhang, Z.; Liao, Q.; et al. Flexible and printable paper-based strain sensors for wearable and large-area green electronics. Nanoscale 2016, 8, 13025-32.
21. Su, T.; Liu, N.; Lei, D.; et al. Flexible MXene/bacterial cellulose film sound detector based on piezoresistive sensing mechanism. ACS. Nano. 2022, 16, 8461-71.
22. Fan, X.; Chen, J.; Yang, J.; Bai, P.; Li, Z.; Wang, Z. L. Ultrathin, rollable, paper-based triboelectric nanogenerator for acoustic energy harvesting and self-powered sound recording. ACS. Nano. 2015, 9, 4236-43.
23. Pi, Y.; Liu, Q.; Li, Z.; et al. Scalable and eco-friendly flexible loudspeakers for distributed human-machine interactions. npj. Flex. Electron. 2023, 7, 45.
24. Zhang, Y.; Zhang, B.; Lv, Y.; Wang, P.; Liu, T.; Meng, C. Flexible and breathable MXene-modified paper-based piezoresistive pressure sensors integrated into airbag pillow for sleep monitoring. Soft. Sci. 2025, 5, 17.
25. Im, H.; Jo, E.; Kang, Y.; Kim, J. Versatile foldable inkjet‐printed thermoacoustic loudspeaker on paper. Adv. Funct. Mater. 2024, 35, 2415218.
26. Quintana, R.; Qiu, X.; Benjamin, A.; Panicker, P.; Schmidt, G.; Hübler, A. C. The printed piezoelectric loudspeaker model: a modal perspective from system identification. Appl. Acoust. 2021, 174, 107722.
27. Shehzad, M.; Wang, S.; Pan, H.; Jaffari, G.; Wang, Y. PVDF based ferroelectric membrane origami loudspeaker. Sens. Actuators. A. Phys. 2023, 360, 114539.
28. Salimpoor, V. N.; Benovoy, M.; Larcher, K.; Dagher, A.; Zatorre, R. J. Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nat. Neurosci. 2011, 14, 257-62.
29. Vuust, P.; Heggli, O. A.; Friston, K. J.; Kringelbach, M. L. Music in the brain. Nat. Rev. Neurosci. 2022, 23, 287-305.
31. Kim, H. S.; Kim, H. S. Effect of a musical instrument performance program on emotional intelligence, anxiety, and aggression in Korean elementary school children. Psychol. Music. 2017, 46, 440-53.
32. Liu, X.; Xu, Y.; Alter, K.; Tuomainen, J. Emotional connotations of musical instrument timbre in comparison with emotional speech prosody: evidence from acoustics and event-related potentials. Front. Psychol. 2018, 9, 737.
33. Turchet, L.; Stefani, D.; Pauwels, J. Musician-AI partnership mediated by emotionally-aware smart musical instruments. Int. J. Hum. Comput. Stud. 2024, 191, 103340.
34. Bhandarkar, A.; Gemicioglu, T.; Dwivedi, B.; Seim, C.; Starner, T. Learning piano songs with passive haptic training: an interactive lesson. In UbiComp/ISWC '22: The 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Cambridge, United Kingdom; Association for Computing Machinery, 2022; pp. 10-2.
35. Calandra, D.; De Lorenzis, F.; Cannavò, A.; Lamberti, F. Immersive virtual reality and passive haptic interfaces to improve procedural learning in a formal training course for first responders. Virtual. Reality. 2022, 27, 985-1012.
36. Pescara, E.; Polly, T.; Schankin, A.; Beigl, M. Reevaluating passive haptic learning of morse code. In UbiComp '19: The 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing, London, United Kingdom; Association for Computing Machinery, 2019; pp. 186-94.
37. Luzhnica, G.; Veas, E.; Seim, C. Passive haptic learning for vibrotactile skin reading. In UbiComp '18: The 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Singapore, Singapore; Association for Computing Machinery, 2018; pp. 40-3.
38. Furuya, S.; Oku, T.; Nishioka, H.; Hirano, M. Surmounting the ceiling effect of motor expertise by novel sensory experience with a hand exoskeleton. Sci. Robot. 2025, 10, eadn3802.
39. Fang, L.; Müller, T.; Pescara, E.; Fischer, N.; Huang, Y.; Beigl, M. Investigating passive haptic learning of piano songs using three tactile sensations of vibration, stroking and tapping. Proc. ACM. Interact. Mob. Wearable. Ubiquitous. Technol. 2023, 7, 1-19.
40. Huang, K.; Do, E. Y.; Starner, T. PianoTouch: a wearable haptic piano instruction system for passive learning of piano skills. In 2008 12th IEEE International Symposium on Wearable Computers, Pittssburgh, USA, Sep 28 - Oct 01, 2008; IEEE, 2008; pp. 41-4.
41. Gemicioglu, T.; Hopper, E.; Dwivedi, B.; et al. Passive haptic rehearsal for augmented piano learning in the wild. Proc. ACM. Interact. Mob. Wearable. Ubiquitous. Technol. 2024, 8, 1-26.
42. Wang, N.; Zhang, H.; Qiu, X.; et al. Recent advances in ferroelectret fabrication, performance optimization, and applications. Adv. Mater. 2024, 36, e2400657.
43. Galikhanov, M. F.; Zhigaeva, I. A.; Minnakhmetova, A. K.; Deberdeev, R. Y. Biodegradability of electret polymer materials. Russ. J. Appl. Chem. 2008, 81, 1258-61.
44. Guzhova, A. A.; Galikhanov, M. F.; Kuznetsova, N. V.; Petrov, V. A.; Khairullin, R. Z. Effect of polylactic acid crystallinity on its electret properties. AIP. Conf. Proc. 2016, 1767, 020009.
45. Qiu, W.; Zhong, J.; Jiang, T.; et al. A low voltage-powered soft electromechanical stimulation patch for haptics feedback in human-machine interfaces. Biosens. Bioelectron. 2021, 193, 113616.
Cite This Article
Open Access
How to Cite
Download Citation
Export Citation File:
Type of Import
Tips on Downloading Citation
Citation Manager File Format
Type of Import
Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.
Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.
About This Article
Copyright
Data & Comments
Data
0
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at [email protected].