Dr. Jerry A. Smith

Elevating data quality in accumulation modeling is paramount for #cyber CAT modeling advancement. In part 3 of our cyber #knowledge series Timothy Marshall and Stephan Brunner call for enhanced exposure #data for improved cyber #accumulation modeling: https://ow.ly/2sc250REcmO #Insurance

13

Excellent report from General Dynamics Information Technology John P. Curran regarding adoption and trends of AI in the Govt. I used Meta to generate summary below and included link to original article. Main AI use cases are chat and creating/updating computer code. Note also the 23% failure rate of AI projects due to user expectation misalignment. This can be overcome with thorough protoyping and establishing use case metrics. - Successful AI project completion timelines are narrowing, with an average of 14 months from pilot to production. - Future demand for AI projects is building around chat and coding-related functions. - 54% of surveyed AI experts said projects currently under development involve improving computer code. - 59% of projects involve "research and understanding", which may relate to creating chat functions. - 23% of AI projects fail due to misalignment between AI solutions and user expectations. - Partnership efforts are required for successful AI projects. Data diversity is crucial, and incorporating diverse data sources can lead to better AI outcomes. Trends: - AI adoption is increasing across the Federal government, with a growing focus on chat and coding-related functions. - Agencies are leveraging cloud-native services, tooling, and other capabilities to accelerate AI project development. - There is a growing interest in translating older code to newer languages, such as Python or cloud-based analytic tools. - Domain-specific chat functionality is becoming a priority, with many agencies planning to implement chat services in the next two years. Statistics: - 325 AI experts surveyed across the Federal government - 14 months: average time from pilot to production for AI projects - 54%: percentage of projects currently under development that involve improving computer code - 59%: percentage of projects that involve "research and understanding" - 23%: percentage of AI projects that fail due to misalignment between AI solutions and user expectations. Bill Casola John Grim Leslie Anne McIndoe Kim Giffin Michael S. Freeman III Tobias Quaranta https://lnkd.in/eqvmAVFG

3

We have published a Commentary from Isaac Wink! 💬 Back to Basics: Considering Categories of Data Services Consults https://lnkd.in/egRbrNfr "Consultations are fundamental to data librarianship, serving as a vital means of one-on-one support for researchers. However, the topics and forms of support unique to data services consults are not always carefully considered. This commentary addresses five common services offered by data librarians...and considers strategies for successful patron support within the boundaries of a consultation." #DataLibs #DataServices #Consultations

2

Librarian of Congress Expands DMCA Exemption for Text and Data Mining DENYING "Class 4: New exemption for computer programs, for purposes of good-faith artificial intelligence (“AI”) trustworthiness research." "The Office received a petition for a new exemption to examine one aspect of these implications—AI trustworthiness. The proposed exemption would cover the circumvention of safeguards and contract terms on online platforms hosting AI models for the purpose of researching biased and other undesirable outputs. After reviewing the rulemaking record, the Register concludes that an exemption is not necessary to allow this socially important research and recommends denying the petition."   https://lnkd.in/eip3cg-v

12

1 Comment

Using deterministic NLP models in conjunction with Phi-3 to summarize long form documents, speeding up the review process and improving accuracy. #NLP #Phi3 #LLM

1

What has ARPA-H done with its first billion dollars of awards? Follow this link to see the awardees to its Open BAA program, which funded investigator-initiated projects in select areas. Do the Republicans (Cassody etc.) have a point with this being government waste (and warranting a merger ARPA-H with other NIH institutes and a 1B budget cut)? Or is this the future of hard-hitting goal directed science? https://lnkd.in/e7mrSP44

4

1 Comment

💡 Interesting observations (part 1) on LLM output safety evaluations by BreezeML’s Founding Engineer, Hunter Zhang, Ph.D.! #artificialintelligence #ai #ml #governance #aigovernance #responsibleai #grc #compliance #aicompliance #airegulation #law #regulation #machinelearning #technology #breezeml #evaluations #llm #safety #aisafety

7

In a new post on the ALI Insights blog, Sara Schapiro is joined by Kate Tromble of the Data Quality Campaign to explain how the NEED Act would modernize Statewide Longitudinal Data Systems. Better data ➡ better outcomes for kids! https://lnkd.in/eP5G3znk

3

In his latest blog post for GWSC, Sambadi Majumder, PhD shares how he built a user friendly graphical user interface (GUI) for spaciotemporal data visualizations to help the Center's analysis team complete their work more efficiently. https://lnkd.in/eDXXYHUv #water #watersecurity #dataviz #datavisualization #GUI #analysis #dataanalysis #environment #climate

4

1 Comment

Statement from Data Foundation President & CEO Nick Hart, Ph.D. on the The White House Office of Management and Budget's Final Rule on Public Trust in Data: "OMB’s release of the final Public Trust rule marks an important milestone in implementing the #EvidenceAct. The success of this rule will depend on meaningful collaboration across the federal data ecosystem, far beyond recognized statistical agencies and units. The Data Foundation urges statistical agencies to work closely with Chief Data Officers, Evaluation Officers, Performance Officers, Chief Information Officers, Chief Financial Officers, and other key data leaders to ensure this rule supports a comprehensive, government-wide approach to evidence-building activities and responsible data use. The Data Foundation looks forward to engaging with our community and partners to fully understand the implications of the regulation rule and support its effective implementation." #CDO #Performance #GPRA #EvidenceBased #Privacy #CIPSEA #FSS #Statistics #CFO #CIO cc Karin Orvis Sam Berger Dominic Mancini Rob Friedlander Steve Pierson https://lnkd.in/eXQGwdta

24

https://buff.ly/3L4F2gn #Webinar: Dissecting the nuances of various #data sovereignty #legislation, exploring #BestPractices for model training and inference pipelines that meet these requirements, and identifying critical considerations for #AI teams #PureStorage #DataPlatform #DataArchitecture

1

The recent release (August 14, 2024) of the Congressional Research Service report, "Oversight of Laboratory Biosafety and Biosecurity: Current Policies and Options for Congress" (https://lnkd.in/eY4z3U-v), highlights concerns that #artificialintelligence (AI) could be repurposed or misused in biological design, raising significant biosafety and biosecurity issues. Similarly, RAND has discussed the U.S. Department of Homeland Security proposed collaborative, whole-of-government strategy for developing and adopting AI initiatives, as outlined in their "2024 Artificial Intelligence (AI) Roadmap" (https://lnkd.in/eJQs9Xe2). Both CRS and RAND emphasize that the rapid pace of AI development and integration is fueling growing concerns around "cultural, ethical, and privacy issues." 📢 While the potential of AI #technology is vast, there’s a critical need to understand the 2nd and 3rd order impacts through modeling & simulation (M&S), and rigorous testing. By embracing AI responsibly, DHS and other federal organizations can not only drive innovation but also address the social concerns that these transformative technologies may bring like United States Department of Defense investing in data and AI structure outlined in their "DOD #Innovation Fact Sheet" ( https://lnkd.in/eJC7FtMq )

7

In this peer-reviewed article in the journal Nature, the authors find that LLMs are unreliable even for simple tasks and that this lack of reliability INCREASES as the model scales, gets shaped-up, and becomes more instructable. Worse yet, the shaped-up models score as highly on Avoidance measures, yet increase their ability to provide plausible-enough answers. Researchers find there is danger as the level of complexity of an answer increases, humans without requisite domain expertise will be unable to ascertain that the LLM is providing an unreliable/false answer. You can see why this would be an issue. We're effectively creating our own Sophon issue. (And shame on you if you haven't read The Remembrance of Time trilogy, or at least The Three Body Problem.) Throw all the data centers you want at the problem. It isn't a compute issue. And it isn't a model issue - researchers tested these hypotheses across multiple raw and shaped-up models, including LLAMA and GPT-4. Unless and until we can get LLMs to return a response that says "I don't know" or "I can't do that" we're in danger. But doing so would pierce the veil of illusion that Generative AI vendors and cloud services and GPU companies need to promulgate in order to keep that goldrush going. LLMs are inherently flawed, a dead-end in AI research. The sooner the bubble bursts, the sooner we can move on to a more promising path. #LLM #Research #Hype #GenerativeAI #AI https://lnkd.in/gvrKKAAK

188

52 Comments

Today, we're joined by Shreya Shankar, a PhD student at UC Berkeley Electrical Engineering & Computer Sciences (EECS) to discuss DocETL, a declarative system for building and optimizing LLM-powered data processing pipelines for complex document analysis tasks. We explore DocETL's optimizer architecture, the intricacies of building agentic systems for data processing, current benchmarks for data tasks vs. reasoning tasks, and the importance of robust evaluation methods for human-in-the-loop LLM workflows. Additionally, we cover the significance of effective validation prompts, building fault-tolerant agentic systems, and the need for benchmarks tailored to LLM-powered data processing tasks. Plus, Shreya shares real-world applications and the future directions for DocETL. 🎧 / 🎥 Listen or watch the full episode on our page: https://twimlai.com/go/703.

18

1 Comment

🌐 We've talked a lot about real-time intelligence requiring a #software-first mindset, but what does this mean in context? How is #AI fundamentally transforming the #geospatialintelligence industry? Get the primer on the shift to space-based #intelligence and why it's crucial for your organization to not just act fast, but first: https://lnkd.in/e7ViVUxS #spacebasedintelligence #artificialintelligence #GEOINT

65

A new report out of analyst firm Intersect360 Research is shedding some light on just how valuable the #HPC and AI market is. Taking both of these #technologies as a singular unit, Intersect360 Research found that the HPC-AI market size reached $85.7 billion in 2023. https://ow.ly/O2Hv50RCMB0 #artificialintelligence

13