Windows 應用程式中的凝視互動與眼動追蹤

提供支援，根據使用者眼睛的位置與移動來追蹤其目光、注意力與存在感。

重要API： Windows.Devices.Input.Preview、 GazeDevicePreview、 GazePointPreview、 GazeInputSourcePreview

概觀

凝視輸入是一種強大的互動與使用 Windows 應用程式的方式，對於患有神經肌肉疾病（如 ALS）及其他肌肉或神經功能受損的使用者來說，特別適合作為輔助技術。

此外，凝視輸入同樣為遊戲（包括目標獲取與追蹤）以及傳統生產力應用程式、自助服務機及其他互動場景提供極具吸引力的機會，這些場景中傳統輸入裝置（鍵盤、滑鼠、觸控）無法使用，或是讓使用者能釋放雙手以完成其他任務（如提購物袋）會很有幫助。

隱私權

由於眼動追蹤裝置可能收集到敏感個人資料，您必須在應用程式清單中申報 gazeInput 該功能（詳見以下 設定 章節）。 當宣告時，Windows 會自動提示使用者一個同意對話框（應用程式首次執行時），使用者必須授權應用程式與眼動追蹤裝置通訊並存取這些資料。

此外，若您的應用程式收集、儲存或轉移眼動追蹤資料，您必須在應用程式的隱私聲明中說明此點，並遵守應用程式開發者協議及 Microsoft 商店政策中關於個人資訊的所有其他要求。

設定

要在你的 Windows 應用程式中使用凝視輸入 API，你需要：

• 請在應用程式 manifest 檔中指定 gazeInput 功能。

  使用 Visual Studio 清單設計器開啟 Package.appxmanifest 檔案，或手動選擇檢視程式碼，並在節點中插入以下DeviceCapabilityCapabilities功能：

  <Capabilities>
     <DeviceCapability Name="gazeInput" />
  </Capabilities>
  

• 一個支援 Windows 的眼動追蹤裝置，連接到你的系統（內建或周邊設備）並開啟。

  請參閱「 Windows 10 眼動控制的開始 」，了解支援的眼動追蹤裝置清單。

基本的眼動追蹤

在此範例中，我們示範如何在 Windows 應用程式中追蹤使用者的視線，並利用時間功能搭配基本的點擊測試，顯示使用者維持對特定元素凝視焦點的能力。

使用一個小橢圓標示應用程式視窗中的視點位置，而 Windows 社群工具包中的 RadialProgressBar 則隨機放置在畫布上。 當進度條偵測到凝視焦點時，計時器會啟動，進度條會在進度條達到100%時隨機移位到畫布上。

使用計時器取樣的凝視追蹤

從 Gaze 輸入範例（基本）下載此範例

1. 首先，我們設定了使用者介面（MainPage.xaml）。

   <Page
       x:Class="gazeinput.MainPage"
       xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
       xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
       xmlns:local="using:gazeinput"
       xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
       xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
       xmlns:controls="using:Microsoft.Toolkit.Uwp.UI.Controls"    
       mc:Ignorable="d">
   
       <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
           <Grid x:Name="containerGrid">
               <Grid.RowDefinitions>
                   <RowDefinition Height="Auto"/>
                   <RowDefinition Height="*"/>
               </Grid.RowDefinitions>
               <StackPanel x:Name="HeaderPanel" 
                       Orientation="Horizontal" 
                       Grid.Row="0">
                   <StackPanel.Transitions>
                       <TransitionCollection>
                           <AddDeleteThemeTransition/>
                       </TransitionCollection>
                   </StackPanel.Transitions>
                   <TextBlock x:Name="Header" 
                          Text="Gaze tracking sample" 
                          Style="{ThemeResource HeaderTextBlockStyle}" 
                          Margin="10,0,0,0" />
                   <TextBlock x:Name="TrackerCounterLabel"
                          VerticalAlignment="Center"                 
                          Style="{ThemeResource BodyTextBlockStyle}"
                          Text="Number of trackers: " 
                          Margin="50,0,0,0"/>
                   <TextBlock x:Name="TrackerCounter"
                          VerticalAlignment="Center"                 
                          Style="{ThemeResource BodyTextBlockStyle}"
                          Text="0" 
                          Margin="10,0,0,0"/>
                   <TextBlock x:Name="TrackerStateLabel"
                          VerticalAlignment="Center"                 
                          Style="{ThemeResource BodyTextBlockStyle}"
                          Text="State: " 
                          Margin="50,0,0,0"/>
                   <TextBlock x:Name="TrackerState"
                          VerticalAlignment="Center"                 
                          Style="{ThemeResource BodyTextBlockStyle}"
                          Text="n/a" 
                          Margin="10,0,0,0"/>
               </StackPanel>
               <Canvas x:Name="gazePositionCanvas" Grid.Row="1">
                   <controls:RadialProgressBar
                       x:Name="GazeRadialProgressBar" 
                       Value="0"
                       Foreground="Blue" 
                       Background="White"
                       Thickness="4"
                       Minimum="0"
                       Maximum="100"
                       Width="100"
                       Height="100"
                       Outline="Gray"
                       Visibility="Collapsed"/>
                   <Ellipse 
                       x:Name="eyeGazePositionEllipse"
                       Width="20" Height="20"
                       Fill="Blue" 
                       Opacity="0.5" 
                       Visibility="Collapsed">
                   </Ellipse>
               </Canvas>
           </Grid>
       </Grid>
   </Page>
   

2. 接著，我們初始化應用程式。

   在此片段中，我們宣告全域物件，並覆寫 OnNavigatedTo 頁面事件以啟動 凝視裝置觀察者 ，並覆寫 OnNavigatedFrom 頁面事件以停止 凝視裝置觀察者。

   using System;
   using Windows.Devices.Input.Preview;
   using Windows.UI.Xaml.Controls;
   using Windows.UI.Xaml;
   using Windows.Foundation;
   using System.Collections.Generic;
   using Windows.UI.Xaml.Media;
   using Windows.UI.Xaml.Navigation;
   
   namespace gazeinput
   {
       public sealed partial class MainPage : Page
       {
           /// <summary>
           /// Reference to the user's eyes and head as detected
           /// by the eye-tracking device.
           /// </summary>
           private GazeInputSourcePreview gazeInputSource;
   
           /// <summary>
           /// Dynamic store of eye-tracking devices.
           /// </summary>
           /// <remarks>
           /// Receives event notifications when a device is added, removed, 
           /// or updated after the initial enumeration.
           /// </remarks>
           private GazeDeviceWatcherPreview gazeDeviceWatcher;
   
           /// <summary>
           /// Eye-tracking device counter.
           /// </summary>
           private int deviceCounter = 0;
   
           /// <summary>
           /// Timer for gaze focus on RadialProgressBar.
           /// </summary>
           DispatcherTimer timerGaze = new DispatcherTimer();
   
           /// <summary>
           /// Tracker used to prevent gaze timer restarts.
           /// </summary>
           bool timerStarted = false;
   
           /// <summary>
           /// Initialize the app.
           /// </summary>
           public MainPage()
           {
               InitializeComponent();
           }
   
           /// <summary>
           /// Override of OnNavigatedTo page event starts GazeDeviceWatcher.
           /// </summary>
           /// <param name="e">Event args for the NavigatedTo event</param>
           protected override void OnNavigatedTo(NavigationEventArgs e)
           {
               // Start listening for device events on navigation to eye-tracking page.
               StartGazeDeviceWatcher();
           }
   
           /// <summary>
           /// Override of OnNavigatedFrom page event stops GazeDeviceWatcher.
           /// </summary>
           /// <param name="e">Event args for the NavigatedFrom event</param>
           protected override void OnNavigatedFrom(NavigationEventArgs e)
           {
               // Stop listening for device events on navigation from eye-tracking page.
               StopGazeDeviceWatcher();
           }
       }
   }
   

3. 接著，我們添加凝視裝置監控方法。

   在 StartGazeDeviceWatcher中，我們呼叫 CreateWatcher ，並宣告眼動追蹤裝置事件監聽器（DeviceAdded、 DeviceUpdate 和 DeviceRemoved）。

   在 DeviceAdded中，我們檢查眼動追蹤裝置的狀態。 如果是可用的裝置，我們會增加裝置數量並啟用凝視追蹤。 詳情請見下一步。

   在 DeviceUpdated中，我們也啟用凝視追蹤，因為當裝置重新校正時，此事件會觸發。

   在 DeviceRemoved中，我們會遞減裝置計數器並移除裝置事件處理程序。

   在 StopGazeDeviceWatcher中，我們關閉了凝視裝置監視器。

    /// <summary>
    /// Start gaze watcher and declare watcher event handlers.
    /// </summary>
    private void StartGazeDeviceWatcher()
    {
        if (gazeDeviceWatcher == null)
        {
            gazeDeviceWatcher = GazeInputSourcePreview.CreateWatcher();
            gazeDeviceWatcher.Added += this.DeviceAdded;
            gazeDeviceWatcher.Updated += this.DeviceUpdated;
            gazeDeviceWatcher.Removed += this.DeviceRemoved;
            gazeDeviceWatcher.Start();
        }
    }

    /// <summary>
    /// Shut down gaze watcher and stop listening for events.
    /// </summary>
    private void StopGazeDeviceWatcher()
    {
        if (gazeDeviceWatcher != null)
        {
            gazeDeviceWatcher.Stop();
            gazeDeviceWatcher.Added -= this.DeviceAdded;
            gazeDeviceWatcher.Updated -= this.DeviceUpdated;
            gazeDeviceWatcher.Removed -= this.DeviceRemoved;
            gazeDeviceWatcher = null;
        }
    }

    /// <summary>
    /// Eye-tracking device connected (added, or available when watcher is initialized).
    /// </summary>
    /// <param name="sender">Source of the device added event</param>
    /// <param name="e">Event args for the device added event</param>
    private void DeviceAdded(GazeDeviceWatcherPreview source, 
        GazeDeviceWatcherAddedPreviewEventArgs args)
    {
        if (IsSupportedDevice(args.Device))
        {
            deviceCounter++;
            TrackerCounter.Text = deviceCounter.ToString();
        }
        // Set up gaze tracking.
        TryEnableGazeTrackingAsync(args.Device);
    }

    /// <summary>
    /// Initial device state might be uncalibrated, 
    /// but device was subsequently calibrated.
    /// </summary>
    /// <param name="sender">Source of the device updated event</param>
    /// <param name="e">Event args for the device updated event</param>
    private void DeviceUpdated(GazeDeviceWatcherPreview source,
        GazeDeviceWatcherUpdatedPreviewEventArgs args)
    {
        // Set up gaze tracking.
        TryEnableGazeTrackingAsync(args.Device);
    }

    /// <summary>
    /// Handles disconnection of eye-tracking devices.
    /// </summary>
    /// <param name="sender">Source of the device removed event</param>
    /// <param name="e">Event args for the device removed event</param>
    private void DeviceRemoved(GazeDeviceWatcherPreview source,
        GazeDeviceWatcherRemovedPreviewEventArgs args)
    {
        // Decrement gaze device counter and remove event handlers.
        if (IsSupportedDevice(args.Device))
        {
            deviceCounter--;
            TrackerCounter.Text = deviceCounter.ToString();

            if (deviceCounter == 0)
            {
                gazeInputSource.GazeEntered -= this.GazeEntered;
                gazeInputSource.GazeMoved -= this.GazeMoved;
                gazeInputSource.GazeExited -= this.GazeExited;
            }
        }
    }

4. 在這裡，我們檢查該裝置是否在 中 IsSupportedDevice 可行，若可行，嘗試在 中 TryEnableGazeTrackingAsync啟用凝視追蹤。

   在 TryEnableGazeTrackingAsync中，我們宣告凝視事件處理程序，並呼叫 GazeInputSourcePreview.GetForCurrentView（） 以取得輸入來源的參考（必須在 UI 執行緒中呼叫，詳見 保持 UI 執行緒響應式）。

   備註

   只有當已連接且你的應用程式需要相容的眼動追蹤裝置時，才應該呼叫 GazeInputSourcePreview.GetForCurrentView()。 否則，同意對話就沒必要了。

    /// <summary>
    /// Initialize gaze tracking.
    /// </summary>
    /// <param name="gazeDevice"></param>
    private async void TryEnableGazeTrackingAsync(GazeDevicePreview gazeDevice)
    {
        // If eye-tracking device is ready, declare event handlers and start tracking.
        if (IsSupportedDevice(gazeDevice))
        {
            timerGaze.Interval = new TimeSpan(0, 0, 0, 0, 20);
            timerGaze.Tick += TimerGaze_Tick;

            SetGazeTargetLocation();

            // This must be called from the UI thread.
            gazeInputSource = GazeInputSourcePreview.GetForCurrentView();

            gazeInputSource.GazeEntered += GazeEntered;
            gazeInputSource.GazeMoved += GazeMoved;
            gazeInputSource.GazeExited += GazeExited;
        }
        // Notify if device calibration required.
        else if (gazeDevice.ConfigurationState ==
                    GazeDeviceConfigurationStatePreview.UserCalibrationNeeded ||
                    gazeDevice.ConfigurationState ==
                    GazeDeviceConfigurationStatePreview.ScreenSetupNeeded)
        {
            // Device isn't calibrated, so invoke the calibration handler.
            System.Diagnostics.Debug.WriteLine(
                "Your device needs to calibrate. Please wait for it to finish.");
            await gazeDevice.RequestCalibrationAsync();
        }
        // Notify if device calibration underway.
        else if (gazeDevice.ConfigurationState == 
            GazeDeviceConfigurationStatePreview.Configuring)
        {
            // Device is currently undergoing calibration.  
            // A device update is sent when calibration complete.
            System.Diagnostics.Debug.WriteLine(
                "Your device is being configured. Please wait for it to finish"); 
        }
        // Device is not viable.
        else if (gazeDevice.ConfigurationState == GazeDeviceConfigurationStatePreview.Unknown)
        {
            // Notify if device is in unknown state.  
            // Reconfigure/recalbirate the device.  
            System.Diagnostics.Debug.WriteLine(
                "Your device is not ready. Please set up your device or reconfigure it."); 
        }
    }

    /// <summary>
    /// Check if eye-tracking device is viable.
    /// </summary>
    /// <param name="gazeDevice">Reference to eye-tracking device.</param>
    /// <returns>True, if device is viable; otherwise, false.</returns>
    private bool IsSupportedDevice(GazeDevicePreview gazeDevice)
    {
        TrackerState.Text = gazeDevice.ConfigurationState.ToString();
        return (gazeDevice.CanTrackEyes &&
                    gazeDevice.ConfigurationState == 
                    GazeDeviceConfigurationStatePreview.Ready);
    }

5. 接著，我們設置了凝視事件處理程式。

   我們分別在GazeEntered中顯示和在GazeExited中隱藏視線追蹤橢圓。

   在 GazeMoved中，我們根據 GazeEnteredPreviewEventArgs 的 CurrentPoint 所提供的 EyeGazePosition 來移動視線追蹤橢圓。 我們也會管理 RadialProgressBar 上的凝視聚焦計時器，這會觸發進度條的重新定位。 詳情請見下一步。

   /// <summary>
   /// GazeEntered handler.
   /// </summary>
   /// <param name="sender">Source of the gaze entered event</param>
   /// <param name="e">Event args for the gaze entered event</param>
   private void GazeEntered(
       GazeInputSourcePreview sender, 
       GazeEnteredPreviewEventArgs args)
   {
       // Show ellipse representing gaze point.
       eyeGazePositionEllipse.Visibility = Visibility.Visible;
   
       // Mark the event handled.
       args.Handled = true;
   }
   
   /// <summary>
   /// GazeExited handler.
   /// Call DisplayRequest.RequestRelease to conclude the 
   /// RequestActive called in GazeEntered.
   /// </summary>
   /// <param name="sender">Source of the gaze exited event</param>
   /// <param name="e">Event args for the gaze exited event</param>
   private void GazeExited(
       GazeInputSourcePreview sender, 
       GazeExitedPreviewEventArgs args)
   {
       // Hide gaze tracking ellipse.
       eyeGazePositionEllipse.Visibility = Visibility.Collapsed;
   
       // Mark the event handled.
       args.Handled = true;
   }
   
   /// <summary>
   /// GazeMoved handler translates the ellipse on the canvas to reflect gaze point.
   /// </summary>
   /// <param name="sender">Source of the gaze moved event</param>
   /// <param name="e">Event args for the gaze moved event</param>
   private void GazeMoved(GazeInputSourcePreview sender, GazeMovedPreviewEventArgs args)
   {
       // Update the position of the ellipse corresponding to gaze point.
       if (args.CurrentPoint.EyeGazePosition != null)
       {
           double gazePointX = args.CurrentPoint.EyeGazePosition.Value.X;
           double gazePointY = args.CurrentPoint.EyeGazePosition.Value.Y;
   
           double ellipseLeft = 
               gazePointX - 
               (eyeGazePositionEllipse.Width / 2.0f);
           double ellipseTop = 
               gazePointY - 
               (eyeGazePositionEllipse.Height / 2.0f) - 
               (int)Header.ActualHeight;
   
           // Translate transform for moving gaze ellipse.
           TranslateTransform translateEllipse = new TranslateTransform
           {
               X = ellipseLeft,
               Y = ellipseTop
           };
   
           eyeGazePositionEllipse.RenderTransform = translateEllipse;
   
           // The gaze point screen location.
           Point gazePoint = new Point(gazePointX, gazePointY);
   
           // Basic hit test to determine if gaze point is on progress bar.
           bool hitRadialProgressBar = 
               DoesElementContainPoint(
                   gazePoint, 
                   GazeRadialProgressBar.Name, 
                   GazeRadialProgressBar); 
   
           // Use progress bar thickness for visual feedback.
           if (hitRadialProgressBar)
           {
               GazeRadialProgressBar.Thickness = 10;
           }
           else
           {
               GazeRadialProgressBar.Thickness = 4;
           }
   
           // Mark the event handled.
           args.Handled = true;
       }
   }
   

6. 最後，以下是管理這個應用程式凝視對焦計時器的方法。

   DoesElementContainPoint 檢查視線指標是否在進度條上方。 如果是這樣，它會啟動凝視計時器，並在每次凝視計時器刻度時更新進度條。

   SetGazeTargetLocation 設定進度條的初始位置，若進度條完成（視凝視焦點計時器而定），則將進度條移至隨機位置。

   /// <summary>
   /// Return whether the gaze point is over the progress bar.
   /// </summary>
   /// <param name="gazePoint">The gaze point screen location</param>
   /// <param name="elementName">The progress bar name</param>
   /// <param name="uiElement">The progress bar UI element</param>
   /// <returns></returns>
   private bool DoesElementContainPoint(
       Point gazePoint, string elementName, UIElement uiElement)
   {
       // Use entire visual tree of progress bar.
       IEnumerable<UIElement> elementStack = 
           VisualTreeHelper.FindElementsInHostCoordinates(gazePoint, uiElement, true);
       foreach (UIElement item in elementStack)
       {
           //Cast to FrameworkElement and get element name.
           if (item is FrameworkElement feItem)
           {
               if (feItem.Name.Equals(elementName))
               {
                   if (!timerStarted)
                   {
                       // Start gaze timer if gaze over element.
                       timerGaze.Start();
                       timerStarted = true;
                   }
                   return true;
               }
           }
       }
   
       // Stop gaze timer and reset progress bar if gaze leaves element.
       timerGaze.Stop();
       GazeRadialProgressBar.Value = 0;
       timerStarted = false;
       return false;
   }
   
   /// <summary>
   /// Tick handler for gaze focus timer.
   /// </summary>
   /// <param name="sender">Source of the gaze entered event</param>
   /// <param name="e">Event args for the gaze entered event</param>
   private void TimerGaze_Tick(object sender, object e)
   {
       // Increment progress bar.
       GazeRadialProgressBar.Value += 1;
   
       // If progress bar reaches maximum value, reset and relocate.
       if (GazeRadialProgressBar.Value == 100)
       {
           SetGazeTargetLocation();
       }
   }
   
   /// <summary>
   /// Set/reset the screen location of the progress bar.
   /// </summary>
   private void SetGazeTargetLocation()
   {
       // Ensure the gaze timer restarts on new progress bar location.
       timerGaze.Stop();
       timerStarted = false;
   
       // Get the bounding rectangle of the app window.
       Rect appBounds = Windows.UI.ViewManagement.ApplicationView.GetForCurrentView().VisibleBounds;
   
       // Translate transform for moving progress bar.
       TranslateTransform translateTarget = new TranslateTransform();
   
       // Calculate random location within gaze canvas.
           Random random = new Random();
           int randomX = 
               random.Next(
                   0, 
                   (int)appBounds.Width - (int)GazeRadialProgressBar.Width);
           int randomY = 
               random.Next(
                   0, 
                   (int)appBounds.Height - (int)GazeRadialProgressBar.Height - (int)Header.ActualHeight);
   
       translateTarget.X = randomX;
       translateTarget.Y = randomY;
   
       GazeRadialProgressBar.RenderTransform = translateTarget;
   
       // Show progress bar.
       GazeRadialProgressBar.Visibility = Visibility.Visible;
       GazeRadialProgressBar.Value = 0;
   }
   

另請參閱

Resources

• Windows 社群工具包凝視函式庫

主題範例

• 凝視取樣（基本）（C#）