robota_core package¶
Submodules¶
robota_core.attendance module¶
A module to collect statistics on student attendance.
- exception robota_core.attendance.AttendanceError[source]¶
Bases:
Exception
An error in collecting attendance data.
- class robota_core.attendance.StudentAttendance(robota_config: dict, mock: bool = False)[source]¶
Bases:
object
The student attendance class collects data from an external API about student attendance.
- Parameters
robota_config – A dictionary of information about data sources read from the robota config file.
mock – If True, return mock data instead of getting real data from the data source.
robota_core.ci module¶
Module that defines interactions with a Continuous integration server in order to get build information.
- class robota_core.ci.Build(jenkins_build)[source]¶
Bases:
object
A Build is the result of executing a CI job.
- Variables
number – The number of the build.
result – The result of the build
timestamp – The time at which the build started.
commit_id – The ID of the git commit the build was run on.
branch_name – The git branch of the commit the build was run on.
link – A HTML string linking to the web-page that displays the build on Jenkins.
instruction_coverage – A code coverage result from JaCoCo.
- class robota_core.ci.BuildResult(value)[source]¶
Bases:
Enum
Represents the result of a Jenkins build.
- Aborted = 4¶
- Failure = 3¶
- Gitlab_Timeout = 6¶
- Not_Built = 5¶
- Success = 1¶
- Unstable = 2¶
- class robota_core.ci.CIServer[source]¶
Bases:
ABC
A CIServer is a service from which test results are fetched. All of these are abstract methods implemented by subclasses.
- abstract get_job_by_name(job_name: str) Optional[Job] [source]¶
Get a job by its name. Return None if job not found.
- abstract get_jobs_by_folder(folder_name: str) List[Job] [source]¶
Get all jobs located in a particular folder.
- class robota_core.ci.JenkinsCIServer(ci_source: dict)[source]¶
Bases:
CIServer
With Jenkins it is possible to download all of the jobs from a whole project at once. This is much quicker than getting each job one by one as the API requests are slow. For this reason the JenkinsCIServer class downloads all jobs from a project and then helper methods get jobs from the local cache.
Connects to Jenkins and downloads all jobs. If the jobs are heavily nested in folders, it may be necessary to increase the depth parameter to iteratively fetch the lower level jobs.
- Parameters
ci_source – A dictionary of config info for setting up the JenkinsCIServer.
- get_job_by_name(job_name: str) Optional[Job] [source]¶
Get a job by its name. Return None if job not found.
- get_jobs_by_folder(folder_name: str) List[Job] [source]¶
Get all jobs that were located in a particular folder.
- class robota_core.ci.Job(job_data, project_root)[source]¶
Bases:
object
A job is a series of CI checks. Each time a job is executed it stores the result in a build.
- get_build_by_number(number) Optional[Build] [source]¶
Get build of this job by number, where 1 is the chronologically earliest build of a job. If build is not found, returns None.
- get_first_build(start: datetime, end: datetime) Union[None, Build] [source]¶
Return the first (oldest) build in the time window.
- get_first_successful_build(start: datetime, end: datetime) Union[None, Build] [source]¶
Return the first (oldest) successful build in the time window.
- class robota_core.ci.Test(suite: dict, case: dict)[source]¶
Bases:
object
A representation of the result of a Test.
- Variables
name – The name of the test.
result – The result of the test, PASSED or FAILED.
time – The time that the test ran.
branch – The branch of commit the test was run upon. This is not populated on object creation.
robota_core.commit module¶
Objects and for describing and processing Git commits.
- class robota_core.commit.Commit(commit, commit_source: str, project_url: Optional[str] = None)[source]¶
Bases:
object
An abstract object representing a git commit.
- Variables
created_at – (datetime) Commit creation time.
id – Commit id.
parent_ids – (List[string]) The ids of one or more commit parents.
raw_message – (str) The original commit message
message – (str) The commit message cleaned for HTML display.
merge_commit – (bool) Whether this commit is a merge commit.
- class robota_core.commit.CommitCache(start: datetime, end: datetime, branch: str, commits: List[Commit])[source]¶
Bases:
object
A cache of Commit objects from a specific date range and branch.
- class robota_core.commit.CommitComment(comment_data, source: str)[source]¶
Bases:
object
A comment made on a commit.
- class robota_core.commit.Tag(tag_data, source: str)[source]¶
Bases:
object
A tag is a named pointer to a git commit.
- Variables
name – The name of the tag.
commit_id – The id of the commit that the tag points to.
- robota_core.commit.get_merge_commit(feature_tip: Commit, master_commits: List[Commit]) Optional[Commit] [source]¶
Get merge commit ID for the branch “branch_title”. Given the id of the commit at the tip of a feature branch, find where it merges into the master branch by going through the commits ids on the master branch and looking at their parents.
- Parameters
feature_tip – The Commit at the tip of the feature branch.
master_commits – Commits of master branch, ordered by date, most recent first.
- Return merge_commit
The id of the merge commit if branch was merged else returns None.
robota_core.config_readers module¶
- exception robota_core.config_readers.RobotaConfigLoadError[source]¶
Bases:
Exception
The error raised when there is a problem loading the configuration
- exception robota_core.config_readers.RobotaConfigParseError[source]¶
Bases:
Exception
The error raised when there is a problem parsing the configuration
- robota_core.config_readers.get_config(file_names: Union[str, List[str]], data_source: dict) list [source]¶
The base method of the class. Calls different methods to get the config depending on the config type.
- Parameters
file_names – The names of one or more config files to open.
data_source – Information about the source of the config data. The ‘type’ key specifies the source of the data and other keys are extra information about the source like url or API token.
- Returns
a list of parsed file contents, one list element for each file specified in file_names. If a file is not found, the corresponding list element is set to None.
- robota_core.config_readers.get_data_source_info(robota_config: dict, key: str) Optional[dict] [source]¶
Get the information about the data source specified by ‘key’ from the robota_config dictionary.
- robota_core.config_readers.get_gitlab_config(config_variables: dict) Tuple[Path, str] [source]¶
Get config from a Gitlab repository by logging in using an access token and downloading the files from the repository.
- Parameters
config_variables – required keys/value pairs are: gitlab_url: The URL of the gitlab repository gitlab_project: The full project name of the project containing the config files. gitlab_token: The gitlab access token.
- Returns
the temporary directory containing the config files.
- robota_core.config_readers.get_robota_config(config_path: str, substitution_vars: dict) dict [source]¶
The robota config specifies the source for each data type used by RoboTA. The RoboTA config is always stored locally since it contains API tokens. :param config_path: The path of the robota config file to read. :param substitution_vars: An optional dictionary of values to substitute into the config file.
- robota_core.config_readers.parse_config(config_path: Path) dict [source]¶
Parses a config file to extract the configuration variables from it.
- Parameters
config_path – the full file path to the config file.
- Returns
the config variables read from the file. Return type depends on the file type.
- robota_core.config_readers.process_yaml(yaml_content: dict) dict [source]¶
Do custom string substitution to the dictionary produced from reading a YAML file. This is not part of the core YAML spec. This function replaces instances of ${key_name} in strings nested as values in dicts or lists with the value of the key “key_name” if “key_name” occurs in the root of the dictionary.
- robota_core.config_readers.read_csv_file(csv_path: Union[str, Path]) dict [source]¶
Parse a two column csv file. Return dict with first column as keys and second column as values.
- robota_core.config_readers.read_yaml_file(config_location: Path) dict [source]¶
Read a YAML file into a dictionary
- Parameters
config_location – the path of the config file
- Returns
Key-value pairs from the config file
- robota_core.config_readers.rmtree_error(func, path, _)[source]¶
Error handler for
shutil.rmtree
.If the error is due to an access error (read only file) it attempts to add write permission and then retries.
- robota_core.config_readers.substitute_dict(input_value: object, root_keys: dict) object [source]¶
If input_value is a list or dict, recurse into it trying to find strings. If input_value is a string then substitute any variables that occur as keys in root_keys with the values in root_keys. Variables to be substituted are indicated by a bash like syntax, e.g. ${variable_name}.
- robota_core.config_readers.substitute_keys(robota_config: dict, command_line_args: dict) dict [source]¶
Go through all of the data sources replacing any curly bracketed strings by named variables provided to roboTA as command line arguments. This allows substitution of things like a team name or exercise number into the robota config.
- Parameters
robota_config – The dictionary of data sources loaded from robota-config.yaml.
command_line_args – Command line arguments given to RoboTA.
robota_core.data_server module¶
robota_core.github_tools module¶
- class robota_core.github_tools.GithubServer(setup: dict)[source]¶
Bases:
object
A connection to GitHub. Contains methods for interfacing with the API.
Initialise the connection to the server, getting credentials from the credentials file.
- Parameters
setup – dictionary containing GitHub url and authentication token.
robota_core.gitlab_tools module¶
General methods for interfacing with GitLab via the python-Gitlab library.
- class robota_core.gitlab_tools.GitlabGroup(gitlab_connection: Gitlab, group_name: str)[source]¶
Bases:
object
A group is distinct from a project, a group may contain many projects. Projects contained in a group inherit the members of the containing project.
- class robota_core.gitlab_tools.GitlabServer(url: str, token: str)[source]¶
Bases:
object
A connection to the Gitlab server. Contains methods for interfacing with the API. This is held distinct from the Repository object because it can also be used to interface with an Issue server.
Initialise the connection to the server, getting credentials from the credentials file.
- Parameters
url – url of GitLab server
token – Authentication token for gitlab server.
- open_gitlab_group(group_name: str) GitlabGroup [source]¶
robota_core.issue module¶
Objects and for describing and processing Git Issues.
- class robota_core.issue.GitHubIssueServer(issue_server_source: dict)[source]¶
Bases:
IssueServer
- class robota_core.issue.GitLabIssueServer(issue_source: dict)[source]¶
Bases:
IssueServer
An IssueServer with GitLab as the server.
- class robota_core.issue.Issue(issue, issue_source: str, get_comments=True)[source]¶
Bases:
object
An Issue
- Variables
created_at – (datetime) The time at which the issue was created.
assignee – (string) The person to whom the issue was assigned.
closed_at – (datetime) The time at which the issue was closed.
closed_by – (string) The person who closed the issue.
time_stats – (dict) Estimates and reported time taken to work on the issue.
due_date – (datetime) The time at which issue is due to be completed.
title – (string) The title of the issue.
comments – (List[Comment]) A list of Comments associated with the Issue.
state – (string) Whether the issue is open or closed.
milestone – (string) Any milestone the issue is associated with.
url – (string) A link to the Issue on GitLab.
- get_assignee() Optional[str] [source]¶
Return name of issue assignee
:return If issue has an assignee, returns their name else returns None.
- get_assignment_date() Optional[datetime] [source]¶
Get assignment date for an issue. First checks comments for assignment date and if none is found, returns the issue creation date. If there is more than one assignment date, this method will always return the most recent.
- Returns
The date at which the issue was assigned.
- get_comment_timestamp(key_phrase: str, earliest=False) Optional[datetime] [source]¶
Search for a phrase in the comments of an issue If the phrase exists, return creation time of the comment.
- Parameters
key_phrase – a phrase to search for in a comment on the issue.
earliest – If True, return the earliest comment matching key_phrase, else return most recent comment matching key_phrase.
- Returns
If phrase is present in a comment, return the the time of the comment, else return None
- get_date_of_time_spent_record(key_phrase: str) Union[datetime, str] [source]¶
Determine whether a time spent category has been recorded. The key phrase should appear in a comment to indicate what the time has been spent on.
- Parameters
key_phrase – Phrase to search for, which should have a time record associated with it
- Returns
Last edited time of comment recording time spent
- get_recorded_team_member(key_phrase: str) Union[None, List[str]] [source]¶
Report whether a team member has been recorded using a key phrase for issue. Key phrase should appear at the start of a comment to indicate assignment of sub-team member, code reviewer (etc).
- Parameters
key_phrase – Phrase to search for
- Return team_member_recorded
Str
- get_status(deadline: datetime)[source]¶
Get current status of issue if deadline hasn’t passed, otherwise get last status of issue before the deadline, and save in the issue.state attribute so that it is only calculated once.
- Parameters
deadline –
- Returns
- class robota_core.issue.IssueCache(start: Optional[datetime] = None, end: Optional[datetime] = None, get_comments=True, milestone=None)[source]¶
Bases:
object
A cache of Issue objects from a specific date range.
- class robota_core.issue.IssueComment(comment, source: str)[source]¶
Bases:
object
A comment is a textual field attached to an Issue
- Variables
text – (string) The content of the comment message.
created_at – (datetime) The time a comment was made.
updated_at – (datetime) The most recent time the content of a comment was updated.
- class robota_core.issue.IssueServer[source]¶
Bases:
object
An IssueServer is a service from which Issues are extracted.
- robota_core.issue.get_issue_by_title(issues: List[Issue], title: str) Optional[Issue] [source]¶
If issue with ‘title’ exists in ‘issues’, return the issue, else return None.
- Parameters
issues – A list of Issue objects.
title – An issue title
- Returns
Issue with title == title, else None.
- robota_core.issue.new_issue_server(robota_config: dict) Union[None, IssueServer] [source]¶
A factory method for IssueServers.
robota_core.logic module¶
- robota_core.logic.are_list_items_in_other_list(reference_list: List, query_list: List) List[bool] [source]¶
Check whether items in query_list exist in correct_list.
- Parameters
reference_list – The reference list of items
query_list – The items to check - does this list contain items in reference_list?
- Return items_present
Whether items in the correct list are in query_list (bool)
>>> are_list_items_in_other_list([1, 2, 3], [3, 1, 1]) [True, False, True]
- robota_core.logic.are_lists_equal(list_1: list, list_2: list) List[bool] [source]¶
Elementwise comparison of lists. Return list of booleans, one for each element in the input lists, True if element N in list 1 is equal to element N in list 2, else False.
- robota_core.logic.date_is_before(date1: datetime, date2: datetime) bool [source]¶
If date1 and date2 are provided and date1 is before date2 return True, else return False.
- robota_core.logic.find_commit_in_list(commit_id: str, commits: List[Commit]) Union[None, Commit] [source]¶
Find a Commit in a list of Commits by its ID.
- Parameters
commit_id – The id of the commit to find.
commits – The list of Commits to search.
- Returns
Commit if found, else None.
- robota_core.logic.find_feature_parent(feature_commit: Commit, base_commits: List[Commit]) bool [source]¶
Determine whether the provided feature commit has a commit in the base branch with a common parent.
- Parameters
feature_commit – The feature commit being checked.
base_commits – A list of the base commits, most recent first.
- Returns
True if feature_commit has a common parent with a commit in the base branch else False.
- robota_core.logic.fixup_first_feature_commit(feature_branch_commits: List[Commit], initial_guess_of_first_commit: Commit, merge_commits: List[Commit])[source]¶
Fix-up function to look for merge commits on master branch before the tip of the feature branch. Any commits up to and including a merge commit in the history of a feature branch cannot be the first commit on the feature branch.
- Parameters
feature_branch_commits –
- Returns
- robota_core.logic.fraction_of_lists_equal(list_1: list, list_2: list) float [source]¶
Returns the fraction of list elements are equal when compared elementwise.
- robota_core.logic.get_first_feature_commit(base_commits: List[Commit], feature_commits: List[Commit]) Optional[Commit] [source]¶
Get the first commit on a feature branch. Determine first commit by looking for branching points, described by the parent commits. All parameters are lists of commit IDs ordered from newest to oldest
- Parameters
base_commits – commits from base branch (usually master)
feature_commits – commits from feature branch
- Return first_feature_commit
commit ID of first commit on feature
- robota_core.logic.get_value_from_list_of_dicts(list_of_dicts: List[dict], search_key: str, search_value: int, return_key: str)[source]¶
Given a list of dictionaries, identify the required dictionary which contains the search_key: search_value pair. Return the value in that dictionary associated with return_key.
- robota_core.logic.is_date_before_other_dates(query_date: datetime, deadline: datetime, key_date: datetime) bool [source]¶
Determine whether an action was before the deadline and another key date.
- Parameters
query_date – The date of the action in question, e.g. when was an issue assigned, time estimate set, or due date set.
deadline – The deadline of the action
key_date – The query date should be before this significant date, as well as the deadline e.g. branch creation date
- Returns
True if issue query_date was before deadline and key_date else False.
robota_core.merge_request module¶
- class robota_core.merge_request.MergeRequest(merge_request, source: str)[source]¶
Bases:
object
A Merge Request
- class robota_core.merge_request.MergeRequestCache(start: datetime, end: datetime, merge_requests: List[MergeRequest])[source]¶
Bases:
object
A cache of MergeRequest objects from a specific date range.
- add_merge_request(merge_request: MergeRequest)[source]¶
Add a MergeRequest to a MergeRequestCache.
robota_core.remote_provider module¶
- class robota_core.remote_provider.GithubRemoteProvider(provider_source: dict)[source]¶
Bases:
RemoteProvider
- class robota_core.remote_provider.GitlabRemoteProvider(provider_source: dict)[source]¶
Bases:
RemoteProvider
- class robota_core.remote_provider.RemoteProvider[source]¶
Bases:
object
A remote provider is a cloud provider that a git repository can be synchronised to. Remote providers have some features that a basic git Repository does not including merge requests and teams.
- abstract get_members() Dict[str, str] [source]¶
Get a dictionary of names and corresponding usernames of members of this repository.
- get_merge_requests(start: datetime = datetime.datetime(1970, 1, 1, 0, 0, 1), end: datetime = datetime.datetime(2022, 6, 28, 10, 9, 5, 742067)) List[MergeRequest] [source]¶
- robota_core.remote_provider.new_remote_provider(robota_config: dict) Optional[RemoteProvider] [source]¶
Factory method for RemoteProvider.
robota_core.repository module¶
- class robota_core.repository.Branch(branch, source: str)[source]¶
Bases:
object
An abstract object representing a git branch.
- Variables
id – Name of branch.
id – Commit id that branch points to.
- class robota_core.repository.Diff(diff_info, diff_source: str)[source]¶
Bases:
object
A representation of a git diff between two points in time for a single file in a git repository.
- class robota_core.repository.Event(event_data)[source]¶
Bases:
object
A repository event.
- Variables
date – The date and time of the event.
type – ‘deleted’, ‘pushed to’ or ‘pushed new’
ref_type – The thing the event concerns, ‘tag’, ‘branch’, ‘commit’ etc.
ref_name – The name of the thing the event concerns (branch name or tag name)
commit_id – A commit id associated with ref
- class robota_core.repository.GithubRepository(repository_source: dict)[source]¶
Bases:
Repository
- compare(point_1: str, point_2: str) List[Diff] [source]¶
Compare the state of the repository at two points in time. The points may be branch names, tags or commit ids.
- class robota_core.repository.GitlabRepository(data_source: dict)[source]¶
Bases:
Repository
A Gitlab flavour of a repository.
- Variables
project – A connection to the gitlab repository
- compare(point_1: str, point_2: str) List[Diff] [source]¶
Compare the state of the repository at two points in time. The points may be branch names, tags or commit ids. Point 1 must be chronologically before point 2.
- class robota_core.repository.LocalRepository(commit_source: dict)[source]¶
Bases:
Repository
- compare(point_1: str, point_2: str) List[Diff] [source]¶
Compare the state of the repository at two points in time. The points may be branch names, tags or commit ids.
- class robota_core.repository.Repository(project_url: str)[source]¶
Bases:
object
A place where commits, tags, events, branches and files come from.
- Variables
_branches – A list of Branches associated with this repository.
_events – A list of Events associated with this repository.
_diffs – A dictionary of cached diffs associated with this repository. They are labelled in the form key = point_1 + point_2 where point_1 and point_2 are the commit SHAs or branch names that the diff describes.
- abstract compare(point_1: str, point_2: str) List[Diff] [source]¶
Compare the state of the repository at two points in time. The points may be branch names, tags or commit ids.
- get_branch(name: str) Optional[Branch] [source]¶
Get a Branch from the repository by name. If Branch does not exist, return None.
- get_commits(start: Optional[datetime] = None, end: Optional[datetime] = None, branch: Optional[str] = None) List[Commit] [source]¶
Get issues from the issue provider between the start date and end date.
- abstract get_events() List[Event] [source]¶
Return a list of Events associated with this repository.
- abstract get_file_contents(file_path: str, branch: str = 'master') Optional[bytes] [source]¶
Get the decoded contents of a file from the repository. Works well for text files. Might explode for other file types.
- get_tag(name: str, deadline: Optional[datetime] = None, events: Optional[List[Event]] = None) Optional[Tag] [source]¶
Get a git Tag by name.
- Parameters
name – The name of the tag to get.
deadline – If provided, filters tags such that tags are only returned if they existed at deadline.
events – Events corresponding to the repository, required if deadline is specified.
- Returns
The Tag if found else returns None.
- robota_core.repository.new_repository(robota_config: dict) Optional[Repository] [source]¶
Factory method for Repositories.
robota_core.string_processing module¶
- robota_core.string_processing.append_list_to_dict(dictionary: Dict[str, list], key: str, value: list)[source]¶
If key exists in dictionary then append value to it, else add a new key with value.
- Parameters
dictionary – A dictionary to add key and value to.
key – Dictionary key.
value – A value to append to the dictionary list.
- robota_core.string_processing.build_regex_string(string: str) str [source]¶
Escape some characters and replace * and ? wildcard characters with the python regex equivalents.
- robota_core.string_processing.clean(text: str) str [source]¶
Convert any HTML tags to a string representation so HTML cannot be executed.
- robota_core.string_processing.get_link(url: str, link_text: Union[str, int, float]) str [source]¶
Create link (e.g. to a commit).
- robota_core.string_processing.html_newlines(text: str) str [source]¶
Replace any newline characters in a string with html newline characters.
- robota_core.string_processing.list_to_html_rows(list_of_strings: list) str [source]¶
Join list items with html new lines.
- robota_core.string_processing.markdownify(text: str) str [source]¶
Take text in markdown format and output the formatted text with HTML markup.
- robota_core.string_processing.replace_none(input_list: list, replacement='-') list [source]¶
Sanitise list for display purposes.
- robota_core.string_processing.string_to_datetime(date: Optional[str], datetime_format: str = '%Y-%m-%dT%H:%M:%S.%fZ') Optional[datetime] [source]¶
Convert time string (output from GitLab project attributes) to datetime.
- Parameters
date – A string representing the datetime.
datetime_format – The format of ‘date’.
- Return date
The converted datetime.
>>> string_to_datetime('2017-12-06T08:28:32.000Z', "%Y-%m-%dT%H:%M:%S.%fZ") datetime.datetime(2017, 12, 6, 8, 28, 32)