TrackerMinerFS: Tracker Miner Library Reference Manual

## TrackerMinerFS TrackerMinerFS — Abstract base class for filesystem miners |

## Functions

GQuark | tracker_miner_fs_error_quark ()
TrackerIndexingTree * | tracker_miner_fs_get_indexing_tree ()
TrackerDataProvider * | tracker_miner_fs_get_data_provider ()
gdouble | tracker_miner_fs_get_throttle ()
void | tracker_miner_fs_set_throttle ()
void | tracker_miner_fs_check_file ()
void | tracker_miner_fs_notify_finish ()
const gchar * | tracker_miner_fs_get_urn ()
gchar * | tracker_miner_fs_query_urn ()
gboolean | tracker_miner_fs_has_items_to_process ()

## Properties

TrackerDataProvider * | data-provider | Read / Write / Construct Only
guint | processing-pool-ready-limit | Read / Write / Construct
guint | processing-pool-wait-limit | Read / Write / Construct
GFile * | root | Read / Write / Construct Only
gdouble | throttle | Read / Write

## Signals

void | finished | Run Last
void | finished-root | Run Last
gchar* | move-file | Run Last
gboolean | process-file | Run Last
gboolean | process-file-attributes | Run Last
gchar* | remove-children | Run Last
gchar* | remove-file | Run Last

## Types and Values

struct | TrackerMinerFS
| TrackerMinerFSClass
enum | TrackerMinerFSError
enum | TrackerMinerFSEventType

## Object Hierarchy

        GObject
        ╰── TrackerMiner
            ╰── TrackerMinerFS

## Implemented Interfaces

TrackerMinerFS implements GInitable.

## Includes

    #include <libtracker-miner/tracker-miner.h>

## Description

TrackerMinerFS is an abstract base class for miners that collect data from a filesystem where parent/child relationships need to be inserted into the database correctly with queue management.

All the filesystem crawling and monitoring is abstracted away, leaving to implementations the decisions of what directories/files should it process, and the actual data extraction.

Example creating a TrackerMinerFS with our own file system root and data provider.

First create our class and base it on TrackerMinerFS:

    1
    2
    3
    G_DEFINE_TYPE_WITH_CODE (MyMinerFiles, my_miner_files, TRACKER_TYPE_MINER_FS,
                             G_IMPLEMENT_INTERFACE (G_TYPE_INITABLE,
                                                    my_miner_files_initable_iface_init))

Later in our class creation function, we are supplying the arguments we want. In this case, the 'root' is a GFile pointing to a root URI location (for example 'file:///') and 'data_provider' is a TrackerDataProvider used to enumerate 'root' and return children it finds. If 'data_provider' is `NULL` (the default), then a TrackerFileDataProvider is created automatically.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    // Note that only 'name' is mandatory
    miner = g_initable_new (MY_TYPE_MINER_FILES,
                            NULL,
                            error,
                            "name", "MyMinerFiles",
                            "root", root,
                            "data-provider", data_provider,
                            "processing-pool-wait-limit", 10,
                            "processing-pool-ready-limit", 100,
                            NULL);

## Functions

### tracker_miner_fs_error_quark ()

    GQuark
    tracker_miner_fs_error_quark (void);

Gives the caller the GQuark used to identify TrackerMinerFS errors in GError structures. The GQuark is used as the domain for the error.

#### Returns

the GQuark used for the domain of a GError.

Since: 1.2.

### tracker_miner_fs_get_indexing_tree ()

    TrackerIndexingTree *
    tracker_miner_fs_get_indexing_tree (TrackerMinerFS *fs);

Returns the TrackerIndexingTree which determines what files/directories are indexed by `fs`

#### Parameters

fs | a TrackerMinerFS |

#### Returns

The TrackerIndexingTree holding the indexing configuration.

[transfer none]

### tracker_miner_fs_get_data_provider ()

    TrackerDataProvider *
    tracker_miner_fs_get_data_provider (TrackerMinerFS *fs);

Returns the TrackerDataProvider implementation, which is being used to supply GFile and GFileInfo content to Tracker.

#### Parameters

fs | a TrackerMinerFS |

#### Returns

The TrackerDataProvider supplying content.

[transfer none]

Since: 1.2

### tracker_miner_fs_get_throttle ()

    gdouble
    tracker_miner_fs_get_throttle (TrackerMinerFS *fs);

Gets the current throttle value, see `tracker_miner_fs_set_throttle()` for more details.

#### Parameters

fs | a TrackerMinerFS |

#### Returns

a double representing a value between 0.0 and 1.0.

Since: 0.8

### tracker_miner_fs_set_throttle ()

    void
    tracker_miner_fs_set_throttle (TrackerMinerFS *fs,
                                   gdouble throttle);

Tells the filesystem miner to throttle its operations. A value of 0.0 means no throttling at all, so the miner will perform operations at full speed, 1.0 is the slowest value. With a value of 1.0, the `fs` is typically waiting one full second before handling the next batch of queued items to be processed.

#### Parameters

fs | a TrackerMinerFS |
throttle | a double between 0.0 and 1.0 |

Since: 0.8

### tracker_miner_fs_check_file ()

    void
    tracker_miner_fs_check_file (TrackerMinerFS *fs,
                                 GFile *file,
                                 gint priority,
                                 gboolean check_parents);

Tells the filesystem miner to check and index a file at a given priority, this file must be part of the usual crawling directories of TrackerMinerFS. See `tracker_miner_fs_directory_add()`.

#### Parameters

fs | a TrackerMinerFS |
file | GFile for the file to check |
priority | the priority of the check task |
check_parents | whether to check parents and eligibility or not |

Since: 0.10

### tracker_miner_fs_notify_finish ()

    void
    tracker_miner_fs_notify_finish (TrackerMinerFS *fs,
                                    GTask *task,
                                    const gchar *sparql,
                                    GError *error);

Notifies `fs` that all processing on `file` has been finished, if any error happened during file data processing, it should be passed in `error` , else `sparql` should contain correct SPARQL representing the operation in particular.

This function is expected to be called in reaction to all TrackerMinerFS signals

#### Parameters

fs | a TrackerMinerFS |
task | a GTask obtained in a TrackerMinerFS signal/vmethod |
sparql | Resulting sparql for the given operation, or `NULL` if there is an error. | [nullable]
error | a GError with the error that happened during processing, or `NULL`. |

### tracker_miner_fs_get_urn ()

    const gchar *
    tracker_miner_fs_get_urn (TrackerMinerFS *fs,
                              GFile *file);

If the item exists in the store, this function retrieves the URN for a GFile being currently processed.

If `file` is not being currently processed by `fs` , or doesn't exist in the store yet, `NULL` will be returned.

#### Parameters

fs | a TrackerMinerFS |
file | a GFile obtained in “process-file” |

#### Returns

The URN containing the data associated to `file` , or `NULL`.

[transfer none][nullable]

Since: 0.8

### tracker_miner_fs_query_urn ()

    gchar *
    tracker_miner_fs_query_urn (TrackerMinerFS *fs,
                                GFile *file);

If the item exists in the store, this function retrieves the URN of the given GFile

If `file` doesn't exist in the store yet, `NULL` will be returned.

#### Parameters

fs | a TrackerMinerFS |
file | a GFile |

#### Returns

A newly allocated string with the URN containing the data associated to `file` , or `NULL`.

[transfer full]

Since: 0.10

### tracker_miner_fs_has_items_to_process ()

    gboolean
    tracker_miner_fs_has_items_to_process (TrackerMinerFS *fs);

The `fs` keeps many priority queus for content it is processing. This function returns `TRUE` if the sum of all (or any) priority queues is more than 0. This includes items deleted, created, updated, moved or being written back.

#### Parameters

fs | a TrackerMinerFS |

#### Returns

`TRUE` if there are items to process in the internal queues, otherwise `FALSE`.

Since: 0.10

## Types and Values

### struct TrackerMinerFS

    struct TrackerMinerFS;

Abstract miner implementation to get data from the filesystem.

### TrackerMinerFSClass

    typedef struct {
    	TrackerMinerClass parent;

    	gboolean (* process_file)             (TrackerMinerFS       *fs,
    	                                       GFile                *file,
    					       GTask                *task);
    	void     (* finished)                 (TrackerMinerFS       *fs,
    	                                       gdouble               elapsed,
    	                                       gint                  directories_found,
    	                                       gint                  directories_ignored,
    	                                       gint                  files_found,
    	                                       gint                  files_ignored);
    	gboolean (* process_file_attributes)  (TrackerMinerFS       *fs,
    	                                       GFile                *file,
    					       GTask                *task);
    	void     (* finished_root)            (TrackerMinerFS       *fs,
    	                                       GFile                *root,
    	                                       gint                  directories_found,
    	                                       gint                  directories_ignored,
    	                                       gint                  files_found,
    	                                       gint                  files_ignored);
    	gchar *  (* remove_file)              (TrackerMinerFS       *fs,
    	                                       GFile                *file);
    	gchar *  (* remove_children)          (TrackerMinerFS       *fs,
    	                                       GFile                *file);
    	gchar *  (* move_file)                (TrackerMinerFS       *fs,
    					       GFile                *dest,
    	                                       GFile                *source,
    	                                       gboolean              recursive);

    	gboolean (* filter_event)             (TrackerMinerFS          *fs,
    	                                       TrackerMinerFSEventType  type,
    	                                       GFile                   *file,
    	                                       GFile                   *source_file);

    	/* <Private> */
    	gpointer padding[20];
    } TrackerMinerFSClass;

Prototype for the abstract class, `process_file` must be implemented in the deriving class in order to actually extract data.

#### Members

TrackerMinerClass `parent`; | parent object class |
`process_file` () | Called when the metadata associated to a file is requested. |
`finished` () | Called when all processing has been performed. |
`process_file_attributes` () | Called when the metadata associated with a file's attributes changes, for example, the mtime. |
`finished_root` () | Called when all resources on a particular root URI have been processed. |
`remove_file` () |  |
`remove_children` () |  |
`move_file` () |  |
`filter_event` () |  |
gpointer `padding`[20]; | Reserved for future API improvements. |

### enum TrackerMinerFSError

Possible errors returned when calling creating new objects based on the TrackerMinerFS type and other APIs available with this class.

#### Members

TRACKER_MINER_FS_ERROR_INIT | There was an error during initialization of the object. The specific details are in the message. |

Since: 1.2.

### enum TrackerMinerFSEventType

#### Members

TRACKER_MINER_FS_EVENT_CREATED |  |
TRACKER_MINER_FS_EVENT_UPDATED |  |
TRACKER_MINER_FS_EVENT_DELETED |  |
TRACKER_MINER_FS_EVENT_MOVED |  |

## Property Details

### The `“data-provider”` property

      “data-provider”            TrackerDataProvider *

Data provider populating data, e.g. like GFileEnumerator.

Flags: Read / Write / Construct Only

### The `“processing-pool-ready-limit”` property

      “processing-pool-ready-limit” guint

Maximum number of SPARQL updates that can be merged in a single connection to the store.

Flags: Read / Write / Construct

Allowed values: >= 1

Default value: 1

### The `“processing-pool-wait-limit”` property

      “processing-pool-wait-limit” guint

Maximum number of files that can be concurrently processed by the upper layer.

Flags: Read / Write / Construct

Allowed values: >= 1

Default value: 1

### The `“root”` property

      “root”                     GFile *

Top level URI for our indexing tree and file notify clases.

Flags: Read / Write / Construct Only

### The `“throttle”` property

      “throttle”                 gdouble

Modifier for the indexing speed, 0 is max speed.

Flags: Read / Write

Allowed values: [0,1]

Default value: 0

## Signal Details

### The `“finished”` signal

    void
    user_function (TrackerMinerFS *miner_fs,
                   gdouble         elapsed,
                   guint           directories_found,
                   guint           directories_ignored,
                   guint           files_found,
                   guint           files_ignored,
                   gpointer        user_data)

The ::finished signal is emitted when `miner_fs` has finished all pending processing.

#### Parameters

miner_fs | the TrackerMinerFS |
elapsed | elapsed time since mining was started |
directories_found | number of directories found |
directories_ignored | number of ignored directories |
files_found | number of files found |
files_ignored | number of ignored files |
user_data | user data set when the signal handler was connected. |

Flags: Run Last

Since: 0.8

### The `“finished-root”` signal

    void
    user_function (TrackerMinerFS *miner_fs,
                   GFile          *file,
                   gpointer        user_data)

The ::finished-crawl signal is emitted when `miner_fs` has finished finding all resources that need to be indexed with the root location of `file` . At this point, it's likely many are still in the queue to be added to the database, but this gives some indication that a location is processed.

#### Parameters

miner_fs | the TrackerMinerFS |
file | a GFile |
user_data | user data set when the signal handler was connected. |

Flags: Run Last

Since: 1.2

### The `“move-file”` signal

    gchar*
    user_function (TrackerMinerFS *trackerminerfs,
                   GFile          *arg1,
                   GFile          *arg2,
                   gboolean        arg3,
                   gpointer        user_data)

Flags: Run Last

### The `“process-file”` signal

    gboolean
    user_function (TrackerMinerFS *miner_fs,
                   GFile          *file,
                   GTask          *builder,
                   gpointer        user_data)

The ::process-file signal is emitted whenever a file should be processed, and it's metadata extracted.

`builder` is the TrackerSparqlBuilder where all sparql updates to be performed for `file` will be appended.

This signal allows both synchronous and asynchronous extraction, in the synchronous case `cancellable` can be safely ignored. In either case, on successful metadata extraction, implementations must call `tracker_miner_fs_notify_finish()` to indicate that processing has finished on `file` , so the miner can execute the SPARQL updates and continue processing other files.

#### Parameters

miner_fs | the TrackerMinerFS |
file | a GFile |
builder | a TrackerSparqlBuilder |
cancellable | a GCancellable |
user_data | user data set when the signal handler was connected. |

#### Returns

`TRUE` if the file is accepted for processing, `FALSE` if the file should be ignored.

Flags: Run Last

Since: 0.8

### The `“process-file-attributes”` signal

    gboolean
    user_function (TrackerMinerFS *miner_fs,
                   GFile          *file,
                   GTask          *builder,
                   gpointer        user_data)

The ::process-file-attributes signal is emitted whenever a file should be processed, but only the attribute-related metadata extracted.

`builder` is the TrackerSparqlBuilder where all sparql updates to be performed for `file` will be appended. For the properties being updated, the DELETE statements should be included as well.

This signal allows both synchronous and asynchronous extraction, in the synchronous case `cancellable` can be safely ignored. In either case, on successful metadata extraction, implementations must call `tracker_miner_fs_notify_finish()` to indicate that processing has finished on `file` , so the miner can execute the SPARQL updates and continue processing other files.

#### Parameters

miner_fs | the TrackerMinerFS |
file | a GFile |
builder | a TrackerSparqlBuilder |
cancellable | a GCancellable |
user_data | user data set when the signal handler was connected. |

#### Returns

`TRUE` if the file is accepted for processing, `FALSE` if the file should be ignored.

Flags: Run Last

Since: 0.10

### The `“remove-children”` signal

    gchar*
    user_function (TrackerMinerFS *trackerminerfs,
                   GFile          *arg1,
                   gpointer        user_data)

Flags: Run Last

### The `“remove-file”` signal

    gchar*
    user_function (TrackerMinerFS *miner_fs,
                   GFile          *file,
                   gpointer        user_data)

The ::remove-file signal will be emitted on files that need removal according to the miner configuration (either the files themselves are deleted, or the directory/contents no longer need inspection according to miner configuration and their location.

This operation is always assumed to be recursive, the `children_only` argument will be `TRUE` if for any reason the topmost directory needs to stay (e.g. moved from a recursively indexed directory tree to a non-recursively indexed location).

The `builder` argument can be used to provide additional SPARQL deletes and updates necessary around the deletion of those items. If the return value of this signal is `TRUE`, `builder` is expected to contain all relevant deletes for this operation.

If the return value of this signal is `FALSE`, the miner will apply its default behavior, which is deleting all triples that correspond to the affected URIs.

#### Parameters

miner_fs | the TrackerMinerFS |
file | a GFile |
children_only | TRUE if only the children of `file` are to be deleted |
builder | a TrackerSparqlBuilder |
user_data | user data set when the signal handler was connected. |

#### Returns

`TRUE` if `builder` contains all the necessary operations to delete the affected resources, `FALSE` to let the miner implicitly handle the deletion.

Flags: Run Last

Since: 1.8