emergingtrajectories.facts¶

Facts agent. Similar to knowledge agent but simply provides a list of facts and associated sources.

This abstracts away the fact generation from forecast creation, thus allowing us to test different prompting strategies and LLMs.

Attributes¶

`_DEFAULT_NUM_SEARCH_RESULTS`
`facts_base_system_prompt`
`facts_base_user_prompt`

Classes¶

FactBaseFileCache

The KnowledgeBaseFileCache is a simple file-based cache for web content and local files. The cache stores the original HTML, PDF, or TXT content and tracks when (if ever) an agent actually accessed the content.

Functions¶

`uri_to_local`(→ str)	Convert a URI to a local file name. In this case, we typically will use an MD5 sum.
`clean_citations`(→ str)	The analysis currently contains numerical citations that are likely not in order, or in some cases are not used. We will update the cituations to follow the proper numerical order, and also include the URLs at the very end.

Module Contents¶

emergingtrajectories.facts._DEFAULT_NUM_SEARCH_RESULTS = 10¶

emergingtrajectories.facts.facts_base_system_prompt = Multiline-String¶

Show Value

"""You are a researcher tasked with helping forecast economic and social trends. The title of our research project is: {statement_title}.

The project description is as follows...
{statement_description}

We will provide you with content from reports and web pages that is meant to help with the above. We will ask you to review these documents, create a set of bullet points to inform your thinking. Rather than using bullet points, please list each as F1, F2, F3, etc... So that we can reference it.

The content we provided you contains source numbers in the format 'SOURCE: #'. When you extract facts, please include the citation in square brackets, with the #, like [#], but replace "#" with the actual Source # from the crawled content we are providing you.

For example, if you are referring to a fact that came under --- SOURCE: 3 ---, you would write something like: "Data is already trending to hotter temperatures [3]." Do not include the "#" in the brackets, just the number.

Thus, a bullet point would look like this:
F1: (information) [1]
F2: (information) [1]
F3: (information) [2]

... and so on, where F1, F2, F3, etc. are facts, and [1], [2] are the source documents you are extracting the facts from.
"""

emergingtrajectories.facts.facts_base_user_prompt = Multiline-String¶

Show Value

"""Today's date is {the_date}. We will now provide you with all the content we've managed to collect.

----------------------
{scraped_content}
----------------------

Please think step-by-step by (a) extracting critical bullet points from the above, and (b) share any insights you might have based on the facts.

The content we provided you contains source numbers in the format 'SOURCE: #'. When you extract facts, please include the citation in square brackets, with the #, like [#], but replace "#" with the actual Source # from the crawled content we are providing you.

For example, if you are referring to a fact that came under --- SOURCE: 3 ---, you would write something like: "Data is already trending to hotter temperatures [3]." Do not include the "#" in the brackets, just the actual number.

DO NOT PROVIDE A FORECAST, BUT SIMPLY STATE AND SHARE THE FACTS AND INSIGHTS YOU HAVE GATHERED.
"""

emergingtrajectories.facts.uri_to_local(uri: str) → str¶

Convert a URI to a local file name. In this case, we typically will use an MD5 sum.

Parameters:: uri (str) – The URI to convert.
Returns:: The MD5 sum of the URI.
Return type:: str

emergingtrajectories.facts.clean_citations(assistant_analysis: str, ctr_to_source: dict) → str¶

The analysis currently contains numerical citations that are likely not in order, or in some cases are not used. We will update the cituations to follow the proper numerical order, and also include the URLs at the very end.

Parameters:

assistant_analysis – the analysis text from the assistant
ctr_to_source – the mapping of citation number to source URL

Returns:

the cleaned analysis text, with citations following a proper numerical format and URIs at the end of the analysis

Return type:

str

class emergingtrajectories.facts.FactBaseFileCache(folder_path: str, cache_file: str = 'cache.json', crawler=None)¶

Parameters:

folder_path (str) – The folder where the cache will be stored.
cache_file (str, optional) – The name of the cache file. Defaults to “cache.json”.

root_path¶

root_parsed¶

root_original¶

cache_file¶

cache¶

summarize_new_info_multiple_queries(statement, chatbot, google_api_key, google_search_id, google_search_queries, fileout=None) → str¶

summarize_new_info(statement, chatbot, google_api_key, google_search_id, google_search_query, fileout=None) → str¶

save_state() → None¶: Saves the in-memory changes to the knowledge base to the JSON cache file.

load_cache() → None¶: Loads the cache from the cache file, or creates the relevant files and folders if one does not exist.

in_cache(uri: str) → bool¶

Checks if a URI is in the cache already.

Parameters:: uri (str) – The URI to check.
Returns:: True if the URI is in the cache, False otherwise.
Return type:: bool

update_cache(uri: str, obtained_on: datetime.datetime, last_accessed: datetime.datetime) → None¶

Updates the cache file for a given URI, specifically when it was obtained and last accessed.

Parameters:

uri (str) – The URI to update.
obtained_on (datetime) – The date and time when the content was obtained.
last_accessed (datetime) – The date and time when the content was last accessed.

log_access(uri: str) → None¶

Saves the last accessed time and updates the accessed tracker for a given URI.

Parameters:: uri (str) – The URI to update.

get_unaccessed_content() → list[str]¶

Returns a list of URIs that have not been accessed by the agent.

Returns:: A list of URIs that have not been accessed by the agent.
Return type:: list[str]

force_empty(uri: str) → None¶

Saves an empty file for a given URI. Used when the page is erroring out.

Parameters:: uri (str) – The URI to empty the cache for.

get(uri: str) → str¶

Returns the content for a given URI. If the content is not in the cache, it will be scraped and added to the cache.

Parameters:: uri (str) – The URI to get the content for.
Returns:: The content for the given URI.
Return type:: str

add_content(content: str, uri: str = None) → None¶

Adds content to cache.

Parameters:

content (str) – The content to add to the cache.
uri (str, optional) – The URI to use for the content. Defaults to None, in which case an MD5 sum of the content will be used.

add_content_from_file(filepath: str, uri: str = None) → None¶

Adds content from a text file to the cache.

Parameters:

filepath (str) – The path to the file to add to the cache.
uri (str, optional) – The URI to use for the content. Defaults to None, in which case an MD5 sum of the content will be used.